Alternatives
Products that do what Fixing LLM memory degradation in long coding sessions does
Long-session LLM memory degradation (entropy) is the silent killer of complex coding projects. Models like Gemini, GPT-4, and Claude all suffer from it, leading to hallucinations and lost context. I've developed an open-source protocol that temporarily "fixes" this issue by structuring the dialogue. It's not the final architectural solution, but it’s a proven patch for developers working right now. Looking for feedback from the community on how we can solve this structurally. https://github.com/robertomisuraca-blip/LLM-Entropy-Fix-Prot...
- 1
- 2
- 3
- 4ZL
Zep is a long-term memory store designed for conversational AI applications built using modern LLMs. It handles the storage, summarization, embedding, indexing, and enrichment of chat histories, and offers developers a simple, low-latency API to this data. Chat history storage is an infrastructure challenge all developers and enterprises face as they look to move from prototypes to deploying conversational AI applications that provide rich and intimate experiences to users. Key features include long-term memory persistence, auto-summarization, vector search, auto-token counting, and Python…
2023
- 5
- 6IL
LLM Application development is extremely iterative, more so than any other types of development. This is because in addition to all the activities involved in regular application development, we also need to make the LLM Application accurate and reduce hallucination. To improve performance, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, try different agent architecture, and more. There are thousands of permutations to try. We need to be able to easily experiment with these…
2024 · palico.ai
- 7BM
LLMs forget. The standard fix is RAG — retrieve chunks, stuff them in. It works until it doesn't: irrelevant chunks waste tokens, summaries lose structure, and nothing actually models how memory works. Breathe-memory takes a different approach: associative injection. Before each LLM call, it extracts anchors from the user's message (entities, temporal references, emotional signals), traverses a concept graph via BFS, runs optional vector search, and injects only what's relevant — typically in <60ms. When context fills up, instead of summarizing, it extracts a structured graph: topics,…
Mar 2026 · github.com
- 8
- 9LP
I was not getting good cache utilization when including dynamic context in agent threads. After a lot of experimentation, I found a good pattern that minimizes how often long lived conversation history gets modified while still supporting dynamic context. It has flexible hooks for doing things like truncating or summarizing tool outputs when transitioning messages to the long term history. And I'm seeing >>90% of tokens hitting the cache for my agents despite including a lot of dynamic user context. There are a wide range of agent prompting strategies so I'd love to hear where this library…
Jun 2026 · github.com
- 10MW
2017 · github.com
- 11LM
I built a super easy to integrate memory storage and retrieval system for NodeJS projects because I saw a need for information to be shared and persisted across LLM chat sessions (and many other LLM feature interactions). I tried to keep the barrier to use as low as possible so I included built-in support for major LLMs (GPT, Gemini, and Claude) as well as major vector store providers (Weaviate and Pinecone). The memory store works by ingesting and automatically extracting “memories” (summarized single bits of information) from LLM interactions and vectorizing those. When you want to provide…
Mar 2026 · github.com
- 12
- 13PM
This is my attempt in building a memory that evolves and persist for claude code. My approach is inspired from Zettelkasten method, memories are atomic, connected and dynamic. Existing memories can evolve based on newer memories. In the background it uses LLM to handle linking and evolution. I have only used it with claude code so far, it works well with me but still early stage, so rough edges likely. I'm planning to extend it to other coding agents as I use several different agents during development. Looking for feedbacks!
Jan 2026 · github.com
- 14SB
*Motivation* Hi hackers, I'm Asif. I know we dislike premature standardization, but hear me out. LLM Application development is extremely iterative, more so than most other types of application development. We need a process that allows us to iterate faster. LLM Development is highly iterative due to the activities that come with regular software development, as well as the need to make the LLM Application accurate and reduce hallucination. To improve hallucination, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt…
2024 · github.com
- 15IB
I wanted to share a project I have been working on over the past week. It is a simple local memory system that saves your sessions into Markdown files, which can be viewed later. I developed this after using Claude Mem. I really enjoyed working with it, but it was consuming a lot of RAM, and each Claude session was becoming a major resource hog. I also tried other plugins and MCP solutions, but ran into similar issues, either slow performance or concerns about data being sent elsewhere. Because privacy was a big thing for me, I decided to build my own solution that keeps all data local.
Feb 2026 · github.com
- 16AL
Try it out here: https://labs.refuel.ai/playground Refuel LLM (84.2%) outperforms trained human annotators (80.4%), GPT-3-5-turbo (81.3%), PaLM-2 (82.3%) and Claude (79.3%) across a benchmark of 15 text labeling datasets. It is a Llama-v2-13b base model, trained on over 2500 unique datasets (5.24B tokens) spanning categories such as classification, entity resolution, matching, reading comprehension and information extraction.
2023
- 17
- 18AT
We kept shipping “simple” LLM features that were fluent-but-wrong. After too many postmortems we wrote down the failure patterns and added a small reasoning layer in front of the model. It’s model-agnostic, sits beside your existing stack, and you can implement it from a single PDF (MIT). What’s inside the PDF A problem map of 16 failure modes we kept hitting in real systems (OCR/layout drift, table-to-question mismatches, embedding≠meaning, pre-deploy collapse, etc.). Four lightweight gates you can add today: Knowledge-boundary canaries (empty/adversarial/known-fact probes).…
2025 · github.com
- 19
- 20KP
I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edge devices like the Jetson Orin super nano kit which have unified memory. I haven't benched much, but plan to do so more over time and see if I can make a real use of it as I run local LLMs. Let me know if it makes sense :P (I of course vibed this idea)
Jun 2026 · github.com
- 21NL
Refuel LLM (84.2%) outperforms trained human annotators (80.4%), GPT-3-5-turbo (81.3%), PaLM-2 (82.3%) and Claude (79.3%) across a benchmark of 15 text labeling datasets. It is a Llama-v2-13b base model, trained on over 2500 unique datasets (5.24B tokens) spanning categories such as classification, entity resolution, matching, reading comprehension and information extraction. Here is the interactive demo: https://labs.refuel.ai/playground. Pretty fun to play with!
2023
- 22LI
LLM Inference Calculator — Estimate throughput, latency, TTFT, TPOT, and GPU memory usage for large language model inference. LLM 推理计算器 — 估算大模型推理吞吐量 (throughput)、延迟 (latency)、TTFT、TPOT 与 GPU 显存占用。
9d ago · llm-inference-calculator-delta.vercel.app
- 23MT
Hi, this is Tristan, CPO of XTrace. We are launching a very cool feature that is inspired by Andrey Karpathy's LLM Wiki mindmap. Let everyone who doesn't have enough sessions and markdowns made with claude code be able to visualize their own memory mindmap!
May 2026 · github.com
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →