AI · alternatives · 2026

Agent-cache is a multi-tier caching layer for AI agents that stores LLM responses, tool results, and session state in Valkey or Redis. It works with LangChain, LangGraph, and Vercel AI SDK through framework adapters… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.

1. 1
   Progress AI Observability▲168

   Hi HN! We’re the team behind Fiddler, and we recently built an Agent Cache feature. It works by capturing an LLM request/response pair and, once caching is enabled, serving subsequent matching calls locally so duplicate requests never hit the provider. It’s meant to reduce three pains that show up together in agent development: token cost, feedback-loop latency, and output non-determinism. We’d love to hear your thoughts.

   Aug 2026 · telerik.com · its alternatives →

2. 2
   traceAI▲273

   Open-source LLM tracing that speaks GenAI, not HTTP.

   Apr 2026 · github.com · its alternatives →

3. 3
   Agents.json – OpenAPI Specification for LLMs▲212

   Hey HN, we’re building an open specification that lets agents discover and invoke APIs with natural language, built on the OpenAPI standard. agents.json clearly defines the contract between LLMs and API as a standard that's open, observable, and replicable. Here’s a walkthrough of how it works: https://youtu.be/kby2Wdt2Dtk?si=59xGCDy48Zzwr7ND. There’s 2 parts to this: 1. An agents.json file describes how to link API calls together into outcome-based tools for LLMs. This file sits alongside an OpenAPI file. 2. The agents.json SDK loads agents.json files as tools for an LLM that…

   2025 · github.com · its alternatives →

4. 4
   AI Observability by OpenObserve▲317

   OpenTelemetry-native observability for agents and LLMs

   Aug 2026 · openobserve.ai · its alternatives →

5. 5
   Arch▲314

   Hey HN — I’m Adil from Katanemo (with Salman, Shuguang, and Meiyu) We previously shared an early version of this project as ArchGW. Based on customer feedback, the scope expanded from “LLM routing and model access” into something broader: delivery infrastructure for agentic applications. We renamed it to Plano and reworked the architecture accordingly. The problem On-the-ground AI practitioners will tell you that calling an LLM is not the hard part. The really hard part is delivering agentic applications to production quickly and reliably, then iterating without rewriting system code every…

   2024 · github.com · its alternatives →

6. 6
   A Karpathy-style LLM wiki your agents maintain (Markdown and Git)▲260

   I shipped a wiki layer for AI agents that uses markdown + git as the source of truth, with a bleve (BM25) + SQLite index on top. No vector or graph db yet. It runs locally in ~/.wuphf/wiki/ and you can git clone it out if you want to take your knowledge with you. The shape is the one Karpathy has been circling for a while: an LLM-native knowledge substrate that agents both read from and write into, so context compounds across sessions rather than getting re-pasted every morning. Most implementations of that idea land on Postgres, pgvector, Neo4j, Kafka, and a dashboard. I…

   Apr 2026 · github.com · its alternatives →

7. 7
   Vector Cache▲9

   A Python Library for Efficient LLM Query Caching

   2024 · github.com · its alternatives →

8. 8
   liteLLM Proxy Server: 50+ LLM Models, Error Handling, Caching▲140

   Hello hacker news, I’m the maintainer of liteLLM() - package to simplify input/output to OpenAI, Azure, Cohere, Anthropic, Hugging face API Endpoints: https://github.com/BerriAI/litellm/ We’re open sourcing our implementation of liteLLM proxy: https://github.com/BerriAI/litellm/blob/main/cookbook/proxy-... TLDR: It has one API endpoint /chat/completions and standardizes input/output for 50+ LLM models + handles logging, error tracking, caching, streaming What can liteLLM proxy do? - It’s a central place to…

   2023 · github.com · its alternatives →

9. 9
   Butter – A Behavior Cache for LLMs▲50

   Hi HN! I'm Erik. We built Butter, an LLM proxy that makes agent systems deterministic by caching and replaying responses, so automations behave consistently across runs. - It’s a chat completions compatible endpoint, making it easy to drop into existing agents with a custom base_url - The cache is template-aware, meaning lookups can treat dynamic content (names, addresses, etc.) as variables You can see it in action in this demo where it memorizes tic-tac-toe games: https://www.youtube.com/watch?v=PWbyeZwPjuY Why we built this: before Butter, we were Pig.dev (YC W25), where we…

   Oct 2025 · butter.dev · its alternatives →

10. 10BA
    Butter, a muscle memory cache for LLMs▲23

    Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…

    2025 · docs.butter.dev · its alternatives →

11. 11
    GitHub▲5

    Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets. - GuglielmoCerri/khazad

    Jun 2026 · github.com · its alternatives →

12. 12
    ReliAPI▲87

    Stop losing money on failed OpenAI and Anthropic API calls.

    Dec 2025 · kikuai-lab.github.io · its alternatives →

13. 13
    Oodle.ai – $10 per million agent traces▲31

    Hi HN, we're Kiran and Vijay! Over the past two years, we have built a columnar storage engine for observability: logs, metrics, and traces. Today, it's exciting for us to show what we've built on top of that foundation: LLM Agent Observability. Given how non-deterministic agents are, storing all traces without sampling was critical for us. But these traces tend to be in the MBs, sometimes GBs - we needed to store them inexpensively. We also needed the queries and analyses to be fast. To meet both these goals, we store them in S3 in our own parquet-like file format, and query them using AWS…

    Jul 2026 · oodle.ai · its alternatives →

14. 14
    An LLM response cache that's aware of dynamic data▲17

    Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…

    Jan 2026 · blog.butter.dev · its alternatives →

15. 15
    OpenBrowser-AI▲76

    Your AI agent is burning 6x more tokens than it needs to just to browse the web. We built OpenBrowser MCP to fix that. Most browser MCPs give the LLM dozens of tools: click, scroll, type, extract, navigate. Each call dumps the entire page accessibility tree into the context window. One Wikipedia page? 124K+ tokens. Every. Single. Call. OpenBrowser works differently. It exposes one tool. Your agent writes Python code, and OpenBrowser executes it in a persistent runtime with full browser access. The agent controls what comes back. No bloated page dumps. No wasted tokens. Just the data your…

    Apr 2026 · openbrowser.me · its alternatives →

16. 16
    A local-first memory store for LLM agents (SQLite)▲48

    Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc. - CaviraOSS/LongMemory

    Dec 2025 · github.com · its alternatives →

17. 17
    LLM-Term – Simple Rust-based CLI assist tool▲32

    Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.

    2024 · github.com · its alternatives →

18. 18
    Redcache_AI – open-source memory framework for LLMs and Agents▲11

    While building my AI-powered dating app, I couldn't find a memory backend that was affordable,accessible and efficient. I built Redcache-ai to meet this need. Redcache-ai is also available as a Python package. Happy to receive feedback and answer questions. Note: I am not a native English speaker. Apologies for the typos and grammatical errors.

    2024 · github.com · its alternatives →

19. 19
    Cache-Pot▲7

    A superfast memory layer built for AI agents

    Jul 2026 · cache-pot.thatdevguy.in · its alternatives →

20. 20
    20+ Claude Code agents coordinating on real work (open source)▲53

    Single-agent LLMs suck at long-running complex tasks. We’ve open-sourced a multi-agent orchestrator that we’ve been using to handle long-running LLM tasks. We found that single LLM agents tend to stall, loop, or generate non-compiling code, so we built a harness for agents to coordinate over shared context while work is in progress. How it works: 1. Orchestrator agent that manages task decomposition 2. Sub-agents for parallel work 3. Subscriptions to task state and progress 4. Real-time sharing of intermediate discoveries between agents We tested this on a Putnam-level math problem, but the…

    Feb 2026 · github.com · its alternatives →

21. 21AT
    A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)▲55

    I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…

    2025 · llmapitest.com · its alternatives →

22. 22
    Crowkis▲2

    The smart, Rust-built LLM cache and agent memory layer

    Jun 2026 · crowkis.com · its alternatives →

23. 23
    LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)▲45

    G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…

    2023 · github.com · its alternatives →

24. 24
    OpenClaw LLM Proxy▲4

    Scan every LLM API call for PII and injection attacks

    Jun 2026 · secure-mind-live.github.io · its alternatives →

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →