AI · alternatives · 2026

Butter is an OpenAI-compatible API proxy that caches large language model generations and replays them deterministically when similar requests are made. Designed for developers building agent systems and automation… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.

1. 1
   Butter – A Behavior Cache for LLMs▲50

   Hi HN! I'm Erik. We built Butter, an LLM proxy that makes agent systems deterministic by caching and replaying responses, so automations behave consistently across runs. - It’s a chat completions compatible endpoint, making it easy to drop into existing agents with a custom base_url - The cache is template-aware, meaning lookups can treat dynamic content (names, addresses, etc.) as variables You can see it in action in this demo where it memorizes tic-tac-toe games: https://www.youtube.com/watch?v=PWbyeZwPjuY Why we built this: before Butter, we were Pig.dev (YC W25), where we…

   Oct 2025 · butter.dev · its alternatives →

2. 2
   An LLM response cache that's aware of dynamic data▲17

   Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…

   Jan 2026 · blog.butter.dev · its alternatives →

3. 3
   Muscle-Mem, a behavior cache for AI agents▲226

   Hi HN! Erik here from Pig.dev, and today I'd like to share a new project we've just open sourced: Muscle Mem is an SDK that records your agent's tool-calling patterns as it solves tasks, and will deterministically replay those learned trajectories whenever the task is encountered again, falling back to agent mode if edge cases are detected. Like a JIT compiler, for behaviors. At Pig, we built computer-use agents for automating legacy Windows applications (healthcare, lending, manufacturing, etc). A recurring theme we ran into was that businesses already had RPA (pure-software scripts), and…

   2025 · github.com · its alternatives →

4. 4
   Butter▲848

   All-in-one tool for the smoothest virtual workshops 🧈

   2021 · its alternatives →

5. 5
   Butter.ai▲142

   Your team has Q’s, Butter.ai has A’s.

   2017 · its alternatives →

6. 6
   PromptMule▲129

   Cache-as-a-service for generative AI app developement & prod

   2023 · its alternatives →

7. 7AT
   A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)▲55

   I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…

   2025 · llmapitest.com · its alternatives →

8. 8
   Tokenwise▲143

   A smart LLM proxy that shows where you're overpaying

   Jun 2026 · tokenwisehq.com · its alternatives →

9. 9
   Vector Cache▲9

   A Python Library for Efficient LLM Query Caching

   2024 · github.com · its alternatives →

10. 10
    Core – open source memory graph for LLMs – shareable, user owned▲112

    I keep running in the same problem of each AI app “remembers” me in its own silo. ChatGPT knows my project details, Cursor forgets them, Claude starts from zero… so I end up re-explaining myself dozens of times a day across these apps. The deeper problem 1. Not portable – context is vendor-locked; nothing travels across tools. 2. Not relational – most memory systems store only the latest fact (“sticky notes”) with no history or provenance. 3. Not yours – your AI memory is sensitive first-party data, yet you have no control over where it lives or how it’s queried. Demo video:…

    2025 · github.com · its alternatives →

11. 11
    ReliAPI▲87

    Stop losing money on failed OpenAI and Anthropic API calls.

    Dec 2025 · kikuai-lab.github.io · its alternatives →

12. 12AC
    Agent-cache – Multi-tier LLM/tool/session caching for Valkey and Redis▲18

    Multi-tier exact-match cache for AI agents backed by Valkey or Redis. LLM responses, tool results, and session state behind one connection. Framework adapters for LangChain, LangGraph, and Vercel AI SDK. OpenTelemetry and Prometheus built in. No modules required - works on vanilla Valkey 7+ and Redis 6.2+. Shipped v0.1.0 yesterday, v0.2.0 today with cluster mode. Streaming support coming next. Existing options locked you into one tier (LangChain = LLM only, LangGraph = state only) or one framework. This solves both. npm:…

    Apr 2026 · its alternatives →

13. 13
    A local-first memory store for LLM agents (SQLite)▲48

    Local persistent memory store for LLM applications including claude desktop, github copilot, codex, antigravity, etc. - CaviraOSS/LongMemory

    Dec 2025 · github.com · its alternatives →

14. 14
    Oodle.ai – $10 per million agent traces▲31

    Hi HN, we're Kiran and Vijay! Over the past two years, we have built a columnar storage engine for observability: logs, metrics, and traces. Today, it's exciting for us to show what we've built on top of that foundation: LLM Agent Observability. Given how non-deterministic agents are, storing all traces without sampling was critical for us. But these traces tend to be in the MBs, sometimes GBs - we needed to store them inexpensively. We also needed the queries and analyses to be fast. To meet both these goals, we store them in S3 in our own parquet-like file format, and query them using AWS…

    Jul 2026 · oodle.ai · its alternatives →

15. 15
    We Beat MLPerf: Modern Storage for KV Offload and LLM Training▲35

    High performance storage engine for efficient LLM inference and GPU Training.

    Sep 2026 · theopenlake.com · its alternatives →

16. 16
    SemanticGuard▲7

    Cuts your LLM API costs by 40-70%. One line of code.

    May 2026 · semanticguard.dev · its alternatives →

17. 17
    Feather – a fresh Tcl reimplementation (WASM, Go)▲31

    Hey HN! First time showing something here, but I've been furiously working over the holidays on Feather, a from scratch reimplementation of TCL designed for embedding in modern applications. It's starting out as a faithful reimplementation of TCL without I/O, OOP features, or coroutines. TCL has a special place in my heart because the syntax is so elegant for interactive use, and defining domain specific languages. My motiviation is twofold: faster feedback loops for AI, and moldable software for users. It turns out giving AI agents access to the runtime state of your program makes for…

    Dec 2025 · feather-lang.dev · its alternatives →

18. 18CC
    Codex context bloat? 87% avg reduction on SWE-bench Verified traces▲10

    If you had to build a context window manager in 24h, would you stick to the existing model or come up with something better? Here's what I did: 1. Built a proxy that intercepts Codex's calls to OpenAI and rewrites them on the fly. 2. Replayed 3,807 rounds of SWE-bench Verified traces through it: avg prompt 44k → 6k tokens (-87%). 3. Posted it to HN to get the next reduction applied to my confidence interval — starting with the inevitable "How about accuracy?" npx -y pando-proxy · github.com/human-software-us/pando-proxy

    Apr 2026 · npmjs.com · its alternatives →

19. 19
    Determinstic LLM inference for lowest price Gemma 4, with Windows XP▲9

    Open-weight models, fully deterministic. The same answer every time, byte for byte, at the floor price.

    Sep 2026 · tokendelivery.ai · its alternatives →

20. 20
    Julep: A platform to manage memories, knowledge and tools for LLM apps▲23

    Hi all! We've built our fair share of LLM apps, everything from Shopify agents to real-time clones of "Samantha" from "Her". We started facing a lot of annoyances & issues repetitively with building functional apps. So we sifted through 97K posts from the OpenAI Community to confirm and find people with similar problems (& we did). It turns out that there are a handful of low-level problems that everybody who is building an AI app that works well needs to solve: statefulness to manage conversations interaction interface between multiple agents and multiple users. robust document search…

    2024 · github.com · its alternatives →

21. 21
    Llmbuffer – Python library for cache-optimized LLM conversation history▲7

    I was not getting good cache utilization when including dynamic context in agent threads. After a lot of experimentation, I found a good pattern that minimizes how often long lived conversation history gets modified while still supporting dynamic context. It has flexible hooks for doing things like truncating or summarizing tool outputs when transitioning messages to the long term history. And I'm seeing >>90% of tokens hitting the cache for my agents despite including a lot of dynamic user context. There are a wide range of agent prompting strategies so I'd love to hear where this library…

    Jun 2026 · github.com · its alternatives →

22. 22
    Redcache_AI – open-source memory framework for LLMs and Agents▲11

    While building my AI-powered dating app, I couldn't find a memory backend that was affordable,accessible and efficient. I built Redcache-ai to meet this need. Redcache-ai is also available as a Python package. Happy to receive feedback and answer questions. Note: I am not a native English speaker. Apologies for the typos and grammatical errors.

    2024 · github.com · its alternatives →

23. 23AL
    A lightweight LLM proxy to get structured results from most LLMs▲13

    Hey HN! After struggling with complex prompt engineering and unreliable parsing, we built L1M, a simple API that lets you extract structured data from unstructured text and images. curl -X POST https://api.l1m.io/structured \ -H "Content-Type: application/json" \ -H "X-Provider-Url: demo" \ -H "X-Provider-Key: demo" \ -H "X-Provider-Model: demo" \ -d '{ "input": "A particularly severe crisis in 1907 led Congress to enact the Federal Reserve Act in 1913", "schema": { "type": "object", "properties": { "items": { "type": "array", "items": { "type": "object", "properties": {…

    2025 · l1m.io · its alternatives →

24. 24
    Cuts Long Horizon Inference Costs by 50% via external KV Cache Offload▲22

    Hey HN, we’re the developers of OpenLake, an open source storage engine for offloading LLM KV caches from GPU memory into a shared tier of RAM and NVMe. We built OpenLake because KV caches are outgrowing GPU memory. A single 256K token conversation on Gemma 4 31B produces approximately 43GB of KV state, more than half the memory of an 80GB H100. The problem becomes even harder across a cluster: a prefix cached on one GPU host is unavailable when the next request lands on a different GPU, forcing the new GPU to repeat work the fleet has already completed. Once the KV cache is offloaded,…

    Jul 2026 · github.com · its alternatives →

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →