AI · alternatives · 2026

24 alternatives to EchoCache

Stop paying twice for identical LLM queries

Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.

1
ReliAPI▲87
Stop losing money on failed OpenAI and Anthropic API calls.
Dec 2025 · kikuai-lab.github.io · its alternatives →
2
Vector Cache▲9
A Python Library for Efficient LLM Query Caching
2024 · github.com · its alternatives →
3AC
Agent-cache – Multi-tier LLM/tool/session caching for Valkey and Redis▲18
Multi-tier exact-match cache for AI agents backed by Valkey or Redis. LLM responses, tool results, and session state behind one connection. Framework adapters for LangChain, LangGraph, and Vercel AI SDK. OpenTelemetry and Prometheus built in. No modules required - works on vanilla Valkey 7+ and Redis 6.2+. Shipped v0.1.0 yesterday, v0.2.0 today with cluster mode. Streaming support coming next. Existing options locked you into one tier (LangChain = LLM only, LangGraph = state only) or one framework. This solves both. npm:…
Apr 2026 · its alternatives →
4
SemanticGuard▲7
Cuts your LLM API costs by 40-70%. One line of code.
May 2026 · semanticguard.dev · its alternatives →
5AT
A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)▲55
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
6RC
Reduce ChatGPT costs 10x with distributed cache for LLMs▲22
Hello HN! We're building a caching solution for LLMs (ChatGPT, Claude). By combining cutting-edge approaches, such as edge computing, prompt compression, vectorization, and others - it can reduce your AI bills by up to 10x and significantly lower response times. Key Features: - cost efficiency: our system stores frequent queries, reducing the number of upstream (paid) API calls - fast responses: with various nodes globally, we reduce latency by serving data from the nearest location - scalability: designed to handle increasing loads and data sizes without degrading performance. The cache…
2024 · edgematic.dev · its alternatives →
7
LLM Gatewayfree tier▲296
I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…
2025 · llmgateway.io · its alternatives →
8
Tokenwise▲143
A smart LLM proxy that shows where you're overpaying
Jun 2026 · tokenwisehq.com · its alternatives →
9
PromptMule▲129
Cache-as-a-service for generative AI app developement & prod
2023 · its alternatives →
10
Demystifying Advanced RAG Pipelines▲131
I've built an advanced RAG (Retrieval-Augmented Generation) pipeline from scratch to demystify the complex mechanics of modern LLM-powered Question Answering systems. This repository features: -- An implementation of a sub-question query engine from scratch to answer complex user questions. -- Illustrative explanations that unveil the inner workings of the system. -- An analysis of the challenges I faced while working with the system, like prompt engineering and cost estimation. -- Qualitative comparison with similar frameworks like LlamaIndex, offering a broader perspective. Key Takeaway:…
2023 · github.com · its alternatives →
11
Promptly▲13
An AI Cost Optimization Infrastructure for LLM Applications
Mar 2026 · getpromptly.in · its alternatives →
12
An LLM response cache that's aware of dynamic data▲17
Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…
Jan 2026 · blog.butter.dev · its alternatives →
13
Velvet – Store OpenAI requests in your own DB▲109
Hey HN! We’re Emma and Chris, founders of Velvet (https://www.usevelvet.com). Velvet proxies OpenAI calls and stores the requests and responses in your PostgreSQL database. That way, you can analyze logs with SQL (instead of a clunky UI). You can also set headers to add caching and metadata (for analysis). Backstory: We started by building some more general AI data tools (like a text-to-SQL editor). We were frustrated by the lack of basic LLM infrastructure, so ended up pivoting to focus on the tooling we wanted. So many existing apps, like Helicone, were hard to use as power…
2024 · usevelvet.com · its alternatives →
14
Free LLM API▲86
Access 1 billion tokens per month for free
Apr 2026 · github.com · its alternatives →
15BA
Butter, a muscle memory cache for LLMs▲23
Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…
2025 · docs.butter.dev · its alternatives →
16
Taylor AI▲118
Fine-tune open source LLMs in minutes
2023 · trytaylor.ai · its alternatives →
17
GitHub▲5
Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets. - GuglielmoCerri/khazad
Jun 2026 · github.com · its alternatives →
18
GPTCache – Redis for LLMs▲7
Hey folks, As much as we love GPT-4, it's expensive and can be slow at times. That's why we built GPTCache - a semantic cache for autoregressive LMs - atop the vector database Milvus and SQLite. GPTCache provides several benefits: 1) reduced expenses due to minimizing the number of requests and tokens sent to the LLM service 2) enhanced performance by fetching cached query results directly 3) improved scalability and availability by avoiding rate limits, and 4) a flexible development environment that allows developers to verify their application's features without connecting to LLM APIs or…
2023 · github.com · its alternatives →
19
Butter – A Behavior Cache for LLMs▲50
Hi HN! I'm Erik. We built Butter, an LLM proxy that makes agent systems deterministic by caching and replaying responses, so automations behave consistently across runs. - It’s a chat completions compatible endpoint, making it easy to drop into existing agents with a custom base_url - The cache is template-aware, meaning lookups can treat dynamic content (names, addresses, etc.) as variables You can see it in action in this demo where it memorizes tic-tac-toe games: https://www.youtube.com/watch?v=PWbyeZwPjuY Why we built this: before Butter, we were Pig.dev (YC W25), where we…
Oct 2025 · butter.dev · its alternatives →
20
A private, flat monthly subscription for open-source LLMs▲31
Hey HN! We've run our privacy-focused open-source inference company for a while now, and we're launching a flat monthly subscription similar to Anthropic's. It should work with Cline, Roo, KiloCode, Aider, etc — any OpenAI-compatible API client should do. The rate limits at every tier are higher than the Claude rate limits, so even if you prefer using Claude it can be a helpful backup for when you're rate limited, for a pretty low price. Let me know if you have any feedback!
2025 · synthetic.new · its alternatives →
21LP
Llmswap – Python package to reduce LLM API costs by 50-90% with caching▲12
I built llmswap to solve a problem I kept hitting in hackathons - burning through API credits while testing the same prompts repeatedly during development. It's a simple Python package that provides a unified interface for OpenAI, Anthropic, Google Gemini, and local models (Ollama), with built-in response caching that can cut API costs by 50-90%. Key features: - Intelligent caching with TTL and memory limits - Context-aware caching for multi-user apps - Auto-fallback between providers when one fails - Zero configuration - works with environment variables from llmswap import LLMClient client…
2025 · pypi.org · its alternatives →
22
ThriftyAI▲2
Cut AI costs by 80% with intelligent semantic caching.
Dec 2025 · its alternatives →
23
Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload▲22
Hey HN, we’re the developers of OpenLake, an open source storage engine for offloading LLM KV caches from GPU memory into a shared tier of RAM and NVMe. We built OpenLake because KV caches are outgrowing GPU memory. A single 256K token conversation on Gemma 4 31B produces approximately 43GB of KV state, more than half the memory of an 80GB H100. The problem becomes even harder across a cluster: a prefix cached on one GPU host is unavailable when the next request lands on a different GPU, forcing the new GPU to repeat work the fleet has already completed. Once the KV cache is offloaded,…
Jul 2026 · github.com · its alternatives →
24
A benchmark + latency sim for LLM db queries: ClickHouse / Postgres▲12
Performance comparison of ClickHouse vs PostgreSQL using LLM-style query patterns on realistic aircraft tracking data - GitHub - 514-labs/LLM-query-test: Performance comparison of ClickHouse vs PostgreSQL using LLM-style query patterns on realistic aircraft tracking data
2025 · github.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →