GPTCache – Redis for LLMs
Hey folks, As much as we love GPT-4, it's expensive and can be slow at times. That's why we built GPTCache - a semantic cache for autoregressive LMs - atop the vector database Milvus and SQLite. GPTCache provides several benefits: 1) reduced expenses due to minimizing the number of requests and tokens sent to the LLM service 2) enhanced performance by fetching cached query results directly 3) improved scalability and availability by avoiding rate limits, and 4) a flexible development environment that allows developers to verify their application's features without connecting to LLM APIs or…
GPTCache – Redis for LLMs launched on April 12, 2023 with 7 votes, #1,904 of 4,853 launches that month and more than 50% of that year's launches. The site was still online in September 2026, 3 years on.
What it does
Semantic cache for LLMs. Fully integrated with LangChain and llama_index. - GitHub - zilliztech/GPTCache: Semantic cache for LLMs. Fully integrated with LangChain and llama_index.
In the maker’s words, at launch
Hey folks, As much as we love GPT-4, it's expensive and can be slow at times. That's why we built GPTCache - a semantic cache for autoregressive LMs - atop the vector database Milvus and SQLite. GPTCache provides several benefits: 1) reduced expenses due to minimizing the number of requests and tokens sent to the LLM…
Does a similar job
all alternatives →
- RCReduce ChatGPT costs 10x with distributed cache for LLMs2024 · edgematic.dev · ▲22
Hello HN! We're building a caching solution for LLMs (ChatGPT, Claude). By combining cutting-edge approaches, such as edge computing, prompt compression, vectorization, and others - it can reduce your AI bills by up to 10x and significantly lower response times. Key Features: - cost efficiency: our system stores frequent queries, reducing the number of upstream (paid) API calls - fast responses: with various nodes globally, we reduce latency by serving data from the nearest location - scalability: designed to handle increasing loads and data sizes without degrading performance. The cache…
An LLM response cache that's aware of dynamic dataJan 2026 · blog.butter.dev · ▲17Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…
GitHubJun 2026 · github.com · ▲5Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets. - GuglielmoCerri/khazad
- ACAgent-cache – Multi-tier LLM/tool/session caching for Valkey and RedisApr 2026 · ▲18
Multi-tier exact-match cache for AI agents backed by Valkey or Redis. LLM responses, tool results, and session state behind one connection. Framework adapters for LangChain, LangGraph, and Vercel AI SDK. OpenTelemetry and Prometheus built in. No modules required - works on vanilla Valkey 7+ and Redis 6.2+. Shipped v0.1.0 yesterday, v0.2.0 today with cluster mode. Streaming support coming next. Existing options locked you into one tier (LangChain = LLM only, LangGraph = state only) or one framework. This solves both. npm:…
Butter – A Behavior Cache for LLMsOct 2025 · butter.dev · ▲50Hi HN! I'm Erik. We built Butter, an LLM proxy that makes agent systems deterministic by caching and replaying responses, so automations behave consistently across runs. - It’s a chat completions compatible endpoint, making it easy to drop into existing agents with a custom base_url - The cache is template-aware, meaning lookups can treat dynamic content (names, addresses, etc.) as variables You can see it in action in this demo where it memorizes tic-tac-toe games: https://www.youtube.com/watch?v=PWbyeZwPjuY Why we built this: before Butter, we were Pig.dev (YC W25), where we…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, April 2023
the whole month →
- G4
Hi HN, Today we’re launching GPT-4 answers on Phind.com, a developer-focused search engine that uses generative AI to browse the web and answer technical questions, complete with code examples and detailed explanations. Unlike vanilla GPT-4, Phind feeds in relevant websites and technical documentation, reducing the model’s hallucination and keeping it up-to-date. To use it, simply enable the “Expert” toggle before doing a search. GPT-4 is making a night-and-day difference in terms of answer quality. For a question like “How can I RLHF a LLaMa model”, Phind in Expert mode delivers a…
AI · 2023 · phind.com
