AI · alternatives · 2026
24 alternatives to Vector Cache
A Python Library for Efficient LLM Query Caching
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Vector Cache launched in 2024; newer entries below may have overtaken it.
- 1
Hey HN! I built a proof-of-concept for AI memory using Git instead of vector databases. The insight: Git already solved versioned document management. Why are we building complex vector stores when we could just use markdown files with Git's built-in diff/blame/history? How it works: Memories stored as markdown files in a Git repo Each conversation = one commit git diff shows how understanding evolves over time BM25 for search (no embeddings needed) LLMs generate search queries from conversation context Example: Ask "how has my project evolved?" and it uses git diff to show actual…
2025 · github.com · its alternatives →
- 2
- 3
Hey HN! We're excited to share our new open-source project, Marvin. Marvin is a high-level library for building AI-powered software. We developed it to address the challenges of integrating LLMs into more traditional applications. One of the biggest issues is the fact that LLMs only deal with strings (and conversational strings at that), so using them to process structured data is especially difficult. Marvin introduces a new concept called AI Functions. These look and feel just like regular Python functions: you provide typed inputs, outputs, and docstrings. However, instead of relying on…
2023 · github.com · its alternatives →
- 4
- 5AC
Multi-tier exact-match cache for AI agents backed by Valkey or Redis. LLM responses, tool results, and session state behind one connection. Framework adapters for LangChain, LangGraph, and Vercel AI SDK. OpenTelemetry and Prometheus built in. No modules required - works on vanilla Valkey 7+ and Redis 6.2+. Shipped v0.1.0 yesterday, v0.2.0 today with cluster mode. Streaming support coming next. Existing options locked you into one tier (LangChain = LLM only, LangGraph = state only) or one framework. This solves both. npm:…
Apr 2026 · its alternatives →
- 6
I've built an advanced RAG (Retrieval-Augmented Generation) pipeline from scratch to demystify the complex mechanics of modern LLM-powered Question Answering systems. This repository features: -- An implementation of a sub-question query engine from scratch to answer complex user questions. -- Illustrative explanations that unveil the inner workings of the system. -- An analysis of the challenges I faced while working with the system, like prompt engineering and cost estimation. -- Qualitative comparison with similar frameworks like LlamaIndex, offering a broader perspective. Key Takeaway:…
2023 · github.com · its alternatives →
- 7
Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…
Jan 2026 · blog.butter.dev · its alternatives →
- 8BA
Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…
2025 · docs.butter.dev · its alternatives →
- 9RC
Hello HN! We're building a caching solution for LLMs (ChatGPT, Claude). By combining cutting-edge approaches, such as edge computing, prompt compression, vectorization, and others - it can reduce your AI bills by up to 10x and significantly lower response times. Key Features: - cost efficiency: our system stores frequent queries, reducing the number of upstream (paid) API calls - fast responses: with various nodes globally, we reduce latency by serving data from the nearest location - scalability: designed to handle increasing loads and data sizes without degrading performance. The cache…
2024 · edgematic.dev · its alternatives →
- 10
Hey folks, As much as we love GPT-4, it's expensive and can be slow at times. That's why we built GPTCache - a semantic cache for autoregressive LMs - atop the vector database Milvus and SQLite. GPTCache provides several benefits: 1) reduced expenses due to minimizing the number of requests and tokens sent to the LLM service 2) enhanced performance by fetching cached query results directly 3) improved scalability and availability by avoiding rate limits, and 4) a flexible development environment that allows developers to verify their application's features without connecting to LLM APIs or…
2023 · github.com · its alternatives →
- 11
Recently I've been working on making LLM evaluations fast by using bayesian optimization to select a sensible subset. Bayesian optimization is used because it’s good for exploration / exploitation of expensive black box (paraphrase, LLM). I would love to hear your thoughts and suggestions on this!
2024 · github.com · its alternatives →
- 12VD
Discover, evaluate, and access relevant embeddings in your go-to framework. Skip all the infra for scraping, cleaning, indexing, and updating high-quality embeddings.
2023 · embedding.store · its alternatives →
- 13
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
- 14
While building a chat application I couldn't find find a free and opensource tool to store user sessions. This led to redcache-ai. The tool helps with semantic search, Retrieval Augmented Generation(RAG) and storage. This is an early version undergoing rapid iteration. Happy to answer questions and hear feedback.
2024 · github.com · its alternatives →
- 15
Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc. - CaviraOSS/LongMemory
Dec 2025 · github.com · its alternatives →
- 16
While building my AI-powered dating app, I couldn't find a memory backend that was affordable,accessible and efficient. I built Redcache-ai to meet this need. Redcache-ai is also available as a Python package. Happy to receive feedback and answer questions. Note: I am not a native English speaker. Apologies for the typos and grammatical errors.
2024 · github.com · its alternatives →
- 17
I was not getting good cache utilization when including dynamic context in agent threads. After a lot of experimentation, I found a good pattern that minimizes how often long lived conversation history gets modified while still supporting dynamic context. It has flexible hooks for doing things like truncating or summarizing tool outputs when transitioning messages to the long term history. And I'm seeing >>90% of tokens hitting the cache for my agents despite including a lot of dynamic user context. There are a wide range of agent prompting strategies so I'd love to hear where this library…
Jun 2026 · github.com · its alternatives →
- 18
GitHub▲5Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets. - GuglielmoCerri/khazad
Jun 2026 · github.com · its alternatives →
- 19
Hey HN, We are Zain and Ashish, founders of Vanna AI. We recently embarked on an experiment to see if large language models (specifically LLMs) could help in generating SQL queries for real-world datasets. We initially started this project as a web app but realized that it was most useful and had broadest applicability as a Python package since you can then incorporate it into an existing workflow (Jupyter notebook, Slackbot, etc). We've had some good success with customer datasets but we've generally heard a lot of skepticism so we decided to write a paper about the methodology we're using…
2023 · github.com · its alternatives →
- 20
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com · its alternatives →
- 21
Hi HN! I'm Erik. We built Butter, an LLM proxy that makes agent systems deterministic by caching and replaying responses, so automations behave consistently across runs. - It’s a chat completions compatible endpoint, making it easy to drop into existing agents with a custom base_url - The cache is template-aware, meaning lookups can treat dynamic content (names, addresses, etc.) as variables You can see it in action in this demo where it memorizes tic-tac-toe games: https://www.youtube.com/watch?v=PWbyeZwPjuY Why we built this: before Butter, we were Pig.dev (YC W25), where we…
Oct 2025 · butter.dev · its alternatives →
- 22
EchoCache ▲1Stop paying twice for identical LLM queries
Aug 2026 · echocache.vercel.app · its alternatives →
- 23AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
- 24
Hey there HN! We've just open-sourced Vanna – a Python package that allows you to transform questions into SQL. We've leveraged LLMs to enable you to "ask" databases what you need, bypassing the need to "write" complex SQL. Quick Overview: - "Train" using DDL statements, documentation, or known correct SQL statements. - "Ask" questions in natural language and receive SQL, tables, and charts in return. - Open Source Flexibility: Swap storage mechanisms, customize LLMs, and choose your databases. - Local or Hosted: Operate everything locally or use our hosted version for free (including…
2023 · vanna.ai · its alternatives →
Also compare
- I replaced vector databases with Git for AI memory (PoC) alternatives
- LLM Explorer alternatives
- Marvin – build AI functions that use an LLM as a runtime alternatives
- PromptMule alternatives
- Agent-cache – Multi-tier LLM/tool/session caching for Valkey and Redis alternatives
- Demystifying Advanced RAG Pipelines alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →