AI · alternatives · 2026
24 alternatives to An agent that tunes its own cache
The weekend of last week I built chat.betterdb.com as a RAG over Valkey/Redis/Dragonfly docs. The goal was to eat our own dogfood and test publicly our caching libraries. It also saved me from having to come up with various demo/test scenarios, as I could extend the building in public to the demo. There is a tool-result cache sitting between the SDK and tools. Each call is normalized and then checked before executing. If it hits we return from the cache, and if not, we check the semantic cache, which embeds the prompt and checks with KNN via valkey-search. If the cosine…
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
Hi HN! We’re the team behind Fiddler, and we recently built an Agent Cache feature. It works by capturing an LLM request/response pair and, once caching is enabled, serving subsequent matching calls locally so duplicate requests never hit the provider. It’s meant to reduce three pains that show up together in agent development: token cost, feedback-loop latency, and output non-determinism. We’d love to hear your thoughts.
Aug 2026 · telerik.com · its alternatives →
- 2
Kept▲104Your AI chats, saved as Markdown locally with no cloud
May 2026 · kept.work · its alternatives →
- 3AC
Multi-tier exact-match cache for AI agents backed by Valkey or Redis. LLM responses, tool results, and session state behind one connection. Framework adapters for LangChain, LangGraph, and Vercel AI SDK. OpenTelemetry and Prometheus built in. No modules required - works on vanilla Valkey 7+ and Redis 6.2+. Shipped v0.1.0 yesterday, v0.2.0 today with cluster mode. Streaming support coming next. Existing options locked you into one tier (LangChain = LLM only, LangGraph = state only) or one framework. This solves both. npm:…
Apr 2026 · its alternatives →
- 4
Hey HN! We’re Emma and Chris, founders of Velvet (https://www.usevelvet.com). Velvet proxies OpenAI calls and stores the requests and responses in your PostgreSQL database. That way, you can analyze logs with SQL (instead of a clunky UI). You can also set headers to add caching and metadata (for analysis). Backstory: We started by building some more general AI data tools (like a text-to-SQL editor). We were frustrated by the lack of basic LLM infrastructure, so ended up pivoting to focus on the tooling we wanted. So many existing apps, like Helicone, were hard to use as power…
2024 · usevelvet.com · its alternatives →
- 5
Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…
Jan 2026 · blog.butter.dev · its alternatives →
- 6
Bindly▲76Hi HN, I’m a product designer by background, not a traditional software engineer. Over the last year, tools like Claude and ChatGPT completely changed how I work. I started with small internal tools, and now I’m “vibe coding” multiple highly personalized apps. As I iterated between Claude Code (implementation) and ChatGPT (ideation / thinking), I kept running into the same problem. To think clearly, I had to repeatedly re-explain: - what the code currently does, - what changed recently, - and why certain decisions were made. That re-summarization step became a real bottleneck. So I…
2016 · bind.ly · its alternatives →
- 7
While building a chat application I couldn't find find a free and opensource tool to store user sessions. This led to redcache-ai. The tool helps with semantic search, Retrieval Augmented Generation(RAG) and storage. This is an early version undergoing rapid iteration. Happy to answer questions and hear feedback.
2024 · github.com · its alternatives →
- 8
Today we released an open, Valkey-native context layer for AI agents as part of our packages at BetterDB (agent memory, semantic + multi-tier caching, typed retrieval) that run on a Valkey instance no matter where it is - no vendor lock-in. We even started provisioning Valkey instances starting today. Packages are shipped on npm and PyPi. Why we made it: BetterDB originally started as a monitoring and observability platform for Valkey, Redis and any RESP compatible db. This is still the core of the product, but in the process of building this, we kept seeing that one of the fastest-growing…
Jun 2026 · github.com · its alternatives →
- 9
Ably Chat▲73Hey HN, Staff Engineer at Ably here. Over the past few months I've been speaking to engineers building AI assistants, copilots, and agentic workflows (over 40 companies at this point), with particular focus on cloud-hosted agents. I expected the hard problems to be in model selection, prompt engineering, and orchestration. Instead, the same infrastructure challenges kept coming up: realtime sync between agents and end clients is surprisingly painful to get right. - Managing and scaling WebSocket or SSE connections between agents and clients - Buffering messages server-side and implementing…
Apr 2026 · ably.com · its alternatives →
- 10RC
Hello HN! We're building a caching solution for LLMs (ChatGPT, Claude). By combining cutting-edge approaches, such as edge computing, prompt compression, vectorization, and others - it can reduce your AI bills by up to 10x and significantly lower response times. Key Features: - cost efficiency: our system stores frequent queries, reducing the number of upstream (paid) API calls - fast responses: with various nodes globally, we reduce latency by serving data from the nearest location - scalability: designed to handle increasing loads and data sizes without degrading performance. The cache…
2024 · edgematic.dev · its alternatives →
- 11CH
Hi, I'm fiiv, and I'm the creator of Cache Horse. I built it because I wanted an easy plug-n-play solution to caching and simplifying HTTP requests - in particular, on frontend. First, I was fetching data like daily weather, historic currency exchange numbers, air quality readings - and many of those APIs have quota limits. And second, since I was already caching them, I thought it would be useful to batch them together - so I built that feature in. I would love to hear your feedback and thoughts on the project. Thanks!
2025 · cache.horse · its alternatives →
- 12
Hi HN! I'm Erik. We built Butter, an LLM proxy that makes agent systems deterministic by caching and replaying responses, so automations behave consistently across runs. - It’s a chat completions compatible endpoint, making it easy to drop into existing agents with a custom base_url - The cache is template-aware, meaning lookups can treat dynamic content (names, addresses, etc.) as variables You can see it in action in this demo where it memorizes tic-tac-toe games: https://www.youtube.com/watch?v=PWbyeZwPjuY Why we built this: before Butter, we were Pig.dev (YC W25), where we…
Oct 2025 · butter.dev · its alternatives →
- 13
A Python Library for Efficient LLM Query Caching
2024 · github.com · its alternatives →
- 14
Have a look at my semantic caching project! It's built to easily integrate in existing LLM workflows, you can use it as a proxy where the cache forwards missed requests without modification to a specified upstream, automatically updating it's cache with the response. You can also use it as a cache-aside cache with a provided python library. It works by computing embedding vectors of input queries, and matches them to seen query + response pairs using a vector store. Everything is in-memory, so it should be blazing fast :)
2025 · github.com · its alternatives →
- 15
I was not getting good cache utilization when including dynamic context in agent threads. After a lot of experimentation, I found a good pattern that minimizes how often long lived conversation history gets modified while still supporting dynamic context. It has flexible hooks for doing things like truncating or summarizing tool outputs when transitioning messages to the long term history. And I'm seeing >>90% of tokens hitting the cache for my agents despite including a lot of dynamic user context. There are a wide range of agent prompting strategies so I'd love to hear where this library…
Jun 2026 · github.com · its alternatives →
- 16BA
Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…
2025 · docs.butter.dev · its alternatives →
- 17
Hello HN, I built Chatter because I was spending too much time copy-pasting feedback from my Discord channels and GitHub issues into a spreadsheet. I realized I was reading the same bug report 50 times, just phrased differently. The Problem: When you have a community, feedback is fragmented. A user complains on Discord, another opens a GitHub issue, and a third emails support. It’s hard to see the magnitude of a problem when the data is siloed and unstructured. What I built: Chatter is an aggregator that listens to these channels via webhooks/API. instead of just listing the messages,…
Jan 2026 · chatter.plus · its alternatives →
- 18
A superfast memory layer built for AI agents
Jul 2026 · cache-pot.thatdevguy.in · its alternatives →
- 19
Hey folks, As much as we love GPT-4, it's expensive and can be slow at times. That's why we built GPTCache - a semantic cache for autoregressive LMs - atop the vector database Milvus and SQLite. GPTCache provides several benefits: 1) reduced expenses due to minimizing the number of requests and tokens sent to the LLM service 2) enhanced performance by fetching cached query results directly 3) improved scalability and availability by avoiding rate limits, and 4) a flexible development environment that allows developers to verify their application's features without connecting to LLM APIs or…
2023 · github.com · its alternatives →
- 20BO
Hey HN, I'm Kristiyan, former Engineering Manager for Redis' Visual Developer Tools (including Redis Insight). I built BetterDB because Valkey is growing fast but lacks proper observability tooling. BetterDB is a monitoring platform for Valkey (and Redis) that focuses on what existing tools miss: Historical persistence – Slowlog entries disappear when the buffer fills. BetterDB persists them so you can see what queries were running at 3am, which clients were connected, and what anomalies were detected — not just current state. Pattern analysis – Stop scrolling through raw slowlog entries.…
Jan 2026 · its alternatives →
- 21CA
Imagine you have to make a little change in all your microservices, like changing a line in Dockerfile, but you have 50 microservices, ouch right! The PRs, pipelines, reviews... I have created a dashboard with a chatbot where you can simply ask "Update all repositories with alpine base image version X to Y", and then click next a bunch of time and see a list of PRs with their pipeline status. Hi friends I'm Daniel, the creator of Infra as AI. I was laid off from my SRE role a couple of months ago, and decided to build something that could always help me at work. My job was to keep the lights…
2025 · infrastructureas.ai · its alternatives →
- 22
Hey HN, I just spent the last few weeks building a database for agents. Over the last year I built PostHog AI, the company's business analyst agent, where we experimented on giving raw SQL access to PostHog databases vs. exposing tools/MCPs. Needless to say, SQL wins. I left PostHog 3 weeks ago to work on side-projects. I wanted to experiment more with SQL+agents. I built an MVP exposing business data through DuckDB + annotated schemas, and ran a benchmark with 11 LLMs (from Kimi 2.5 to Claude Opus 4.6) answering business questions with either 1) per-source MCP access (e.g. one Stripe…
Apr 2026 · github.com · its alternatives →
- 23
While building my AI-powered dating app, I couldn't find a memory backend that was affordable,accessible and efficient. I built Redcache-ai to meet this need. Redcache-ai is also available as a Python package. Happy to receive feedback and answer questions. Note: I am not a native English speaker. Apologies for the typos and grammatical errors.
2024 · github.com · its alternatives →
- 24
Synthetic data generation is an essential step in training and evaluating LLMs/Agents/RAG pipelines, but tooling around this is still lacking. We're introducing Curator, an open-source library designed to streamline the data curation process. While there are many libraries to prompt LLMs, the semantics of generating synthetic data is different from prompting. For example, we need to process a large number of prompts (sometimes in millions or more) while accepting some failures, utilize several stages of prompting, incorporate human feedback, and filter out bad data using verifiers…
2025 · github.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →