Semcache – I built a semantic cache in Rust
Have a look at my semantic caching project! It's built to easily integrate in existing LLM workflows, you can use it as a proxy where the cache forwards missed requests without modification to a specified upstream, automatically updating it's cache with the response. You can also use it as a cache-aside cache with a provided python library. It works by computing embedding vectors of input queries, and matches them to seen query + response pairs using a vector store. Everything is in-memory, so it should be blazing fast :)
Semcache – I built a semantic cache in Rust launched on June 15, 2025 with 5 votes and more than 75% of that year's launches. The site was still online in September 2026, 1 year on.
What it does
Semantic caching layer for your LLM applications. Reuse responses and reduce token usage. - sensoris/semcache
semcache is a semantic caching layer for your LLM applications. docker run -p 8080:8080 semcache/semcache:latest Configure your application e.g with the OpenAI Python SDK: from openai import OpenAI # Point to your Semcache host instead of OpenAI client = OpenAI ( base_url = "http://localhost:8080" , api_key = "your-key" ) # Cache miss - continues to OpenAI response = client . chat . completions . create ( model = "gpt-4o" , messages = [{ "role" : "user" , "content" : "What is the capital of France?" }] ) # Cache hit - returns instantly response = client . chat . completions . create ( model = "gpt-4o" , messages = [{ "role" : "user" , "content" : "Tell me France's capital city" }] ) Node.js…from github.com
Does a similar job
all alternatives →
Cachet – A drop-in semantic cache for LLM APIs, 100% local, in RustJun 2026 · github.com · ▲5A transparent, 100%-local semantic cache for LLM APIs — drop-in proxy, one line to integrate, written in Rust - abhix2112/Cachet
An LLM response cache that's aware of dynamic dataJan 2026 · blog.butter.dev · ▲17Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…
Cachekit – High performance caching policies library in RustJan 2026 · github.com · ▲47High-performance cache policies and supporting data structures for Rust systems, with optional metrics and benchmarks. - OxidizeLabs/cachekit
FlakeHub Cache: Fast, secure, configurable. A new take on Nix caching2024 · determinate.systems · ▲65Fast, secure, configurable. A bold new take on Nix caching.
- BAButter, a muscle memory cache for LLMs2025 · docs.butter.dev · ▲23
Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 28d ago · world.pixverse.video
Source: Show HN post ↗
Launched alongside, June 2025
the whole month →- IM
Hey HN! Pilots everywhere are required to keep a logbook of all their flying hours, aircraft, airports, and so on. Since I track everything digitally (some people still just use paper logbooks!), I put together some data visualizations and a few 3D globes to show my flying history. This globe is probably my favourite so far: https://jameshard.ing/pilot/globes/all If you’ve got ideas for other graphs or ways to show this kind of data, I’d love to hear them!
Life & fun · 2025 · jameshard.ing
Stunning presentations with AI. No design skills required.
AI · 2025 · chroniclehq.com
Build native mobile apps for iOS and Android without code.
Dev tools · 2025 · bble.io
All your connections, across all your socials.
AI · 2025 · pally.com
I was the main contributor to workout.lol, an open-source fitness app to easily build a workout routine. The project had traction (1.4k GitHub stars, 95 forks, ~20K visits/month), but was eventually sold due to video licensing hurdles. The new owner stopped maintaining it, and the repo went abandoned. Over the next 9 months, I sent 15 emails to try to save it : no replies. Feature requests & issues were ignored. The community was left with a "broken" tool let's say. I couldn't just let it die So I built the new version from scratch with the same open-source spirit, but a better…
Dev tools · 2025 · github.com