AI · alternatives · 2026
24 alternatives to Long term Memory and 50M token window for LLM
Join the discussion on this paper page
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
Built this for my LLM workflows - needed searchable, persistent memory that wouldn't blow up storage costs. I also wanted to use it locally for my research. It's a content-addressed storage system with block-level deduplication (saves 30-40% on typical codebases). I have integrated the CLI tool into most of my workflows in Zed, Claude Code, and Cursor, and I provide the prompt I'm currently using in the repo. The project is in C++ and the build system is rough around the edges but is tested on macOS and Ubuntu 24.04.
2025 · github.com · its alternatives →
- 2
Pick hardware and a model, and feel the real speed of a local LLM before you buy. Simulates tokens/sec and time-to-first-token from verified benchmarks for GPUs, Macs, and more.
6d ago · llmspeed.dev · its alternatives →
- 3
Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc. - CaviraOSS/LongMemory
Dec 2025 · github.com · its alternatives →
- 4
2024 · colab.research.google.com · its alternatives →
- 5ZL
Zep is a long-term memory store designed for conversational AI applications built using modern LLMs. It handles the storage, summarization, embedding, indexing, and enrichment of chat histories, and offers developers a simple, low-latency API to this data. Chat history storage is an infrastructure challenge all developers and enterprises face as they look to move from prototypes to deploying conversational AI applications that provide rich and intimate experiences to users. Key features include long-term memory persistence, auto-summarization, vector search, auto-token counting, and Python…
2023 · its alternatives →
- 6LT
2025 · v0-llm-token-visualizer.vercel.app · its alternatives →
- 7
10M-token BYOK memory. Cut your LLM API costs by 90%.
Feb 2026 · icm-intelligence-layer.vercel.app · its alternatives →
- 8AS
We explored a novel method to gauge the significance of tokens in prompts given to large language models, without needing direct model access. Essentially, we just did an ablation study on the prompt using cosine similarity of the embeddings as the measure. We got surprisingly promising results when comparing this really simple approach to integrated gradients. Curious to hear thoughts from the community!
2023 · heatmap.demos.watchful.io · its alternatives →
- 9
Hi HN, I was once given the advice: Don't waste expensive frontier model credits (GPT/Claude/etc.) on bulk work. Send the boring, repetitive, high-volume jobs to a smaller model, and save the expensive prompts for when you actually need frontier-level reasoning. I complained and told my manager that I shouldnt have to think about using certain models for certain coding tasks, and that one model should handle everything. Well, here we are anyway. If anyone needs a place to absolutely abuse an LLM with high-volume tasks, come beat ours up at https://yolo-auto.com. Here are…
Jul 2026 · yolo-auto.com · its alternatives →
- 10
The standard AI energy debate compares server-side LLM inference to a server-side Google query. I think this misses most of what actually happens on a mobile device during a real search session. I built a parametric model of the full end-to-end mobile search session: 4G/5G radio energy, SoC rendering cost for a 2.5MB page, programmatic advertising RTB auctions running in the background, and network transmission costs for both sides. Then compared it to an equivalent LLM session. Main finding across 10,000 Monte Carlo draws: on mobile, a standard LLM session uses on average 5.4x less…
Apr 2026 · dupr.at · its alternatives →
- 11
Shared persistent memory across all your LLMs.
2025 · teamlayer.xyz · its alternatives →
- 12
- 13
Little tool that I made to understand how (un)reasonable my prompts are.
Jan 2026 · github.com · its alternatives →
- 14
Token-efficiency linter for LLM prompts and payloads - ritenv/tokensift
Aug 2026 · github.com · its alternatives →
- 15BA
Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…
2025 · docs.butter.dev · its alternatives →
- 16
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right now I am running Qwen3-30B-A3B on my 24gb unified memory m4 MacBook Pro at 50 tok/sec and this should definitely not be working for such a large model on my middling hardware. Things are detailed in the README to get up and running and DESIGN.md has details on all the choices and such made along the way.
Aug 2026 · github.com · its alternatives →
- 17
Long-session LLM memory degradation (entropy) is the silent killer of complex coding projects. Models like Gemini, GPT-4, and Claude all suffer from it, leading to hallucinations and lost context. I've developed an open-source protocol that temporarily "fixes" this issue by structuring the dialogue. It's not the final architectural solution, but it’s a proven patch for developers working right now. Looking for feedback from the community on how we can solve this structurally. https://github.com/robertomisuraca-blip/LLM-Entropy-Fix-Prot...
Nov 2025 · github.com · its alternatives →
- 18
Hi everyone, I just released an open source load testing tool for LLMs: https://github.com/twerkmeister/tokenflood === What is it and what problems does it solve? === Tokenflood is a load testing tool for instruction-tuned LLMs hat can simulate arbitrary LLM loads in terms of prompt, prefix, and output lengths and requests per second. Instead of first collecting prompt data for different load types, you can configure the desired parameters for your load test and you are good to go. It also let's you assess the latency effects of potential prompt parameter changes before…
Nov 2025 · github.com · its alternatives →
- 19WM
Pick a Mac and a model, get the memory it needs and the tokens/sec to expect. Covers M1 to M6, every unified memory size, Q4 to FP16. Formula fitted and validated on our own measurements on two machines.
6d ago · macyou.co · its alternatives →
- 20
I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edge devices like the Jetson Orin super nano kit which have unified memory. I haven't benched much, but plan to do so more over time and see if I can make a real use of it as I run local LLMs. Let me know if it makes sense :P (I of course vibed this idea)
Jun 2026 · github.com · its alternatives →
- 21
I built a super easy to integrate memory storage and retrieval system for NodeJS projects because I saw a need for information to be shared and persisted across LLM chat sessions (and many other LLM feature interactions). I tried to keep the barrier to use as low as possible so I included built-in support for major LLMs (GPT, Gemini, and Claude) as well as major vector store providers (Weaviate and Pinecone). The memory store works by ingesting and automatically extracting “memories” (summarized single bits of information) from LLM interactions and vectorizing those. When you want to provide…
Mar 2026 · github.com · its alternatives →
- 22LS
Hi HN, I built llm.sql, an LLM inference framework that reimagines the LLM execution pipeline as a series of structured SQL queries atop SQLite. The motivation: Edge LLMs are getting better, but hardware remains a bottleneck, especially RAM (size and bandwidth). When available memory is less than the model size and KV cache, the OS incurs page faults and swaps pages using LRU-like strategies, resulting in throughput degradation that's hard to notice and even harder to debug. In fact, the memory access pattern during LLM inference is deterministic - we know exactly which weights are needed…
Apr 2026 · its alternatives →
- 23
Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization - RahulSChand/gpu_poor
2023 · github.com · its alternatives →
- 24
i built TXTOS because my models kept forgetting and bluffing. i wanted a portable fix that works across providers without code or setup. TXTOS is a single .txt you paste into any LLM chat. it boots a small reasoning OS that gives you two things by default: a semantic tree memory that survives long threads, and a knowledge boundary guard that pushes back when the model is out of scope. what it is plain text. no scripts, no trackers, no api calls. MIT. the file encodes a protocol for reasoning, memory, and safety. you can diff it and fork it. it is not “a clever prompt”. it behaves like a tiny…
2025 · github.com · its alternatives →
Also compare
- Yet another memory system for LLMs alternatives
- Llmspeed – feel how fast a local LLM runs before you buy the hardware alternatives
- A local-first memory store for LLM agents (SQLite) alternatives
- Long context window for LLMs alternatives
- Zep – Long-Term Memory Store for LLM Apps alternatives
- LLM Token Visualizer – How big is 128k token input alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →