Cachet – A drop-in semantic cache for LLM APIs, 100% local, in Rust
A transparent, 100%-local semantic cache for LLM APIs — drop-in proxy, one line to integrate, written in Rust - abhix2112/Cachet
Cachet – A drop-in semantic cache for LLM APIs, 100% local, in Rust launched on June 23, 2026 with 5 votes, #2,466 of 22,647 launches that month and more than 75% of that year's launches.
What it does
A transparent, 100%-local semantic cache for LLM APIs. Drop it in front of any app with a one-line change, cut the cost of repeated and rephrased calls, and watch the savings add up live. The /__cachet/ dashboard: estimated $ saved ticking up and green “hit” rows streaming in as repeated/rephrased calls are served from cache. Apps hit LLM APIs with the same and nearly-the-same prompts over and over — retries, shared questions, re-runs, slightly reworded queries — and pay full price for every one. The usual answers are either a caching library you wire into your code, or a heavyweight gateway/managed service you stand up and route through. Cachet is neither. It's a single Rust binary you put…from github.com
Does a similar job
all alternatives →
Semcache – I built a semantic cache in Rust2025 · github.com · ▲5Have a look at my semantic caching project! It's built to easily integrate in existing LLM workflows, you can use it as a proxy where the cache forwards missed requests without modification to a specified upstream, automatically updating it's cache with the response. You can also use it as a cache-aside cache with a provided python library. It works by computing embedding vectors of input queries, and matches them to seen query + response pairs using a vector store. Everything is in-memory, so it should be blazing fast :)
An LLM response cache that's aware of dynamic dataJan 2026 · blog.butter.dev · ▲17Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…
GitHubJun 2026 · github.com · ▲5Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets. - GuglielmoCerri/khazad
Cachekit – High performance caching policies library in RustJan 2026 · github.com · ▲47High-performance cache policies and supporting data structures for Rust systems, with optional metrics and benchmarks. - OxidizeLabs/cachekit
LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)2023 · github.com · ▲45G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
Butter – A Behavior Cache for LLMsOct 2025 · butter.dev · ▲50Hi HN! I'm Erik. We built Butter, an LLM proxy that makes agent systems deterministic by caching and replaying responses, so automations behave consistently across runs. - It’s a chat completions compatible endpoint, making it easy to drop into existing agents with a custom base_url - The cache is template-aware, meaning lookups can treat dynamic content (names, addresses, etc.) as variables You can see it in action in this demo where it memorizes tic-tac-toe games: https://www.youtube.com/watch?v=PWbyeZwPjuY Why we built this: before Butter, we were Pig.dev (YC W25), where we…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗
Launched alongside, June 2026
the whole month →
Fundraisly▲1,537AI fundraising agent that finds investors and books meetings
AI · Jun 2026 · fundraisly.com
Homebrew 6.0.0▲1,481Today, I’m proud to announce Homebrew 6.0.0. The most significant changes since 5.1.0 are a new tap trust security mechanism, the new faster, smaller, default internal Homebrew JSON API, sandboxing on Linux, better defaults informed by our user survey, many brew bundle improvements, improved performance and initial support for macOS 27 (Golden Gate). Happy to discuss any questions here!
Dev tools · Jun 2026 · brew.sh
Hi HN! I was super frustrated at apps like tiktok and wechat preventing me from saving or downloading the animated gifs, so I made a little tool to turn a screen recording into a slack-sized emoji. Hope you get some value from this!
Life & fun · Jun 2026 · vorpus.github.io
Search 18 years of Hacker News and chart how any topic, tool, or person trended. Overlay terms to compare their rise and fall across 45M posts and comments.
Life & fun · Jun 2026 · hackernewstrends.com