AI · alternatives · 2026
24 alternatives to Semcache – I built a semantic cache in Rust
Have a look at my semantic caching project! It's built to easily integrate in existing LLM workflows, you can use it as a proxy where the cache forwards missed requests without modification to a specified upstream, automatically updating it's cache with the response. You can also use it as a cache-aside cache with a provided python library. It works by computing embedding vectors of input queries, and matches them to seen query + response pairs using a vector store. Everything is in-memory, so it should be blazing fast :)
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
A transparent, 100%-local semantic cache for LLM APIs — drop-in proxy, one line to integrate, written in Rust - abhix2112/Cachet
Jun 2026 · github.com · its alternatives →
- 2
Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…
Jan 2026 · blog.butter.dev · its alternatives →
- 3
High-performance cache policies and supporting data structures for Rust systems, with optional metrics and benchmarks. - OxidizeLabs/cachekit
Jan 2026 · github.com · its alternatives →
- 4
A Python Library for Efficient LLM Query Caching
2024 · github.com · its alternatives →
- 5
SeekStorm: vector & lexical search - in-process library & multi-tenancy server, in Rust. - SeekStorm/SeekStorm
2024 · github.com · its alternatives →
- 6
Fast, secure, configurable. A bold new take on Nix caching.
2024 · determinate.systems · its alternatives →
- 7BA
Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…
2025 · docs.butter.dev · its alternatives →
- 8
Hello! I got quite some feedback on a web server I'm building, so I'm rewriting the server to be faster and easier to use. I (and maybe some other contributors?) have optimized the web server performance, especially for static file serving and reverse proxying (the last use case I optimized for very recently). I also picked a different configuration format and specification, what I believe is easier to write. Automatic TLS is also enabled by default out of the box, you don't need to even enable it manually, like it was in the original server I was building. Yesterday, I released the first…
2025 · ferron.sh · its alternatives →
- 9
Hey folks, As much as we love GPT-4, it's expensive and can be slow at times. That's why we built GPTCache - a semantic cache for autoregressive LMs - atop the vector database Milvus and SQLite. GPTCache provides several benefits: 1) reduced expenses due to minimizing the number of requests and tokens sent to the LLM service 2) enhanced performance by fetching cached query results directly 3) improved scalability and availability by avoiding rate limits, and 4) a flexible development environment that allows developers to verify their application's features without connecting to LLM APIs or…
2023 · github.com · its alternatives →
- 10FB
2024 · docs.sweep.dev · its alternatives →
- 11
GitHub▲5Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets. - GuglielmoCerri/khazad
Jun 2026 · github.com · its alternatives →
- 12
Hybrid in-memory and disk cache in Rust. Contribute to foyer-rs/foyer development by creating an account on GitHub.
2025 · github.com · its alternatives →
- 13
Fast PUT/GET/DELETE in-memory key-value store for lookaside caching - jonhoo/cucache
2015 · github.com · its alternatives →
- 14
Hi HN! I'm Erik. We built Butter, an LLM proxy that makes agent systems deterministic by caching and replaying responses, so automations behave consistently across runs. - It’s a chat completions compatible endpoint, making it easy to drop into existing agents with a custom base_url - The cache is template-aware, meaning lookups can treat dynamic content (names, addresses, etc.) as variables You can see it in action in this demo where it memorizes tic-tac-toe games: https://www.youtube.com/watch?v=PWbyeZwPjuY Why we built this: before Butter, we were Pig.dev (YC W25), where we…
Oct 2025 · butter.dev · its alternatives →
- 15
I built a Ruby gem for memoization with TTL + LRU cache. It’s thread-safe, and has been helpful in my own apps. Would love to get some feedback: https://github.com/mishalzaman/memo_ttl
2025 · github.com · its alternatives →
- 16CH
Hi, I'm fiiv, and I'm the creator of Cache Horse. I built it because I wanted an easy plug-n-play solution to caching and simplifying HTTP requests - in particular, on frontend. First, I was fetching data like daily weather, historic currency exchange numbers, air quality readings - and many of those APIs have quota limits. And second, since I was already caching them, I thought it would be useful to batch them together - so I built that feature in. I would love to hear your feedback and thoughts on the project. Thanks!
2025 · cache.horse · its alternatives →
- 17
Hi HN, I'm Kyle, the creator of Clipper, a container registry with 7x faster builds via lazy pulls and cache mount exports. In the process of implementing cache mount exports I ended up with a FUSE mounted filesystem backed by a remote content defined store. I went looking for fun things to do with it and found that there's not really a good way to keep Rust incremental build state around on ephemeral runners, so here we are. Rust generally uses https://github.com/mozilla/sccache for caching, but sccache disables itself when incremental compilation is enabled…
Jul 2026 · github.com · its alternatives →
- 18
HTTP client/libcurl TUI front end in Rust, with request + key storage - PThorpe92/CuTE
2023 · github.com · its alternatives →
- 19AC
Multi-tier exact-match cache for AI agents backed by Valkey or Redis. LLM responses, tool results, and session state behind one connection. Framework adapters for LangChain, LangGraph, and Vercel AI SDK. OpenTelemetry and Prometheus built in. No modules required - works on vanilla Valkey 7+ and Redis 6.2+. Shipped v0.1.0 yesterday, v0.2.0 today with cluster mode. Streaming support coming next. Existing options locked you into one tier (LangChain = LLM only, LangGraph = state only) or one framework. This solves both. npm:…
Apr 2026 · its alternatives →
- 20AA
The weekend of last week I built chat.betterdb.com as a RAG over Valkey/Redis/Dragonfly docs. The goal was to eat our own dogfood and test publicly our caching libraries. It also saved me from having to come up with various demo/test scenarios, as I could extend the building in public to the demo. There is a tool-result cache sitting between the SDK and tools. Each call is normalized and then checked before executing. If it hits we return from the cache, and if not, we check the semantic cache, which embeds the prompt and checks with KNN via valkey-search. If the cosine…
May 2026 · its alternatives →
- 21HF
Hey HN, we’re Eliza and Xin. We’re working on Hurry, an open source drop-in tool that adds distributed build caching to Cargo with (almost) zero configuration. Wherever you run cargo build, you can run hurry cargo build instead, and expect around 2-5x (our best benchmark is 22x) faster builds. We built this because we were dissatisfied with the current build caching options available for Rust. Buck and Bazel require learning a new tool. GitHub Actions and swatinem are too coarse-grained (either the whole cache hits, or none of it does) and finicky to debug, and cargo-chef and Docker layer…
Dec 2025 · hurry.build · its alternatives →
- 22
Simple and efficient way to show information about Memcache. - hyperia-sk/memcache-info
2017 · github.com · its alternatives →
- 23
Fence based CAS for Go with read-time validation, stale-write protection, pluggable codecs, and local or Redis backends. - unkn0wn-root/cascache
2025 · github.com · its alternatives →
- 24
Hey HN, we’re the developers of OpenLake, an open source storage engine for offloading LLM KV caches from GPU memory into a shared tier of RAM and NVMe. We built OpenLake because KV caches are outgrowing GPU memory. A single 256K token conversation on Gemma 4 31B produces approximately 43GB of KV state, more than half the memory of an 80GB H100. The problem becomes even harder across a cluster: a prefix cached on one GPU host is unavailable when the next request lands on a different GPU, forcing the new GPU to repeat work the fleet has already completed. Once the KV cache is offloaded,…
Jul 2026 · github.com · its alternatives →
Also compare
- Cachet – A drop-in semantic cache for LLM APIs, 100% local, in Rust alternatives
- An LLM response cache that's aware of dynamic data alternatives
- Cachekit – High performance caching policies library in Rust alternatives
- Vector Cache alternatives
- SeekStorm – open-source sub-millisecond search in Rust alternatives
- FlakeHub Cache: Fast, secure, configurable. A new take on Nix caching alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →