Librarian – Cut token costs by up to 85% for LangGraph and OpenClaw
Hi HN, I'm building Librarian (https://uselibrarian.dev/), an open-source (MIT) context management tool that stops AI agents from burning tokens by blindly re-reading their entire conversation history on every turn. The Problem: If you're building agentic loops in frameworks like LangGraph or OpenClaw, you hit two walls fast: Financial Cost: Token usage scales quadratically over long conversations. Passing the whole history every time gets incredibly expensive. Context Rot: As the context window fills up, the LLM suffers from the "Lost in the Middle" effect. Response latency…
What it does
In the maker’s words, at launch
Hi HN, I'm building Librarian (https://uselibrarian.dev/), an open-source (MIT) context management tool that stops AI agents from burning tokens by blindly re-reading their entire conversation history on every turn. The Problem: If you're building agentic loops in frameworks like LangGraph or OpenClaw, you hit two walls fast: Financial Cost: Token usage scales quadratically over long conversations. Passing the whole history every time gets incredibly expensive. Context Rot: As the context window fills up, the LLM suffers from the "Lost in the Middle" effect. Response latency spikes, and reasoning accuracy drops. The standard workaround is vector search (RAG) over past messages, but that completely loses temporal logic and conversational dependencies. How Librarian Fixes This: We replaced brute-force context windowing with a lightweight reasoning pipeline: Index: After a message, a smaller model asynchronously creates a compressed summary (~100 tokens), building an index of the conversation. Select: When a new prompt arrives, Librarian reads the summary index and reasons about which specific historical messages are actually relevant to the current turn. Hydrate: It fetches only those selected messages and passes them to the responder. The Results: Instead of passing 2,000+ tokens of noise, you pass a highly curated context of ~800 tokens. In our 50-turn benchmarks, this reduces token costs by up to 85% while actually increasing answer accuracy (82% vs 78% for brute-force) because the distracting noise is removed. It currently works as a drop-in integration for LangGraph and OpenClaw. I'd love for you to check out the benchmark suite, try the integrations, and tear the methodology apart. I'll be hanging out in the comments to answer questions, debug, or hear why this approach is terrible. Thanks!
Does the same job
all alternatives →
- TPToken price calculator for 400+ LLMs2024 · github.com · ▲268
Hey HN! Tokencost is a utility library for estimating LLM costs. There are hundreds of different models now, and they all have their own pricing schemes. It’s difficult to keep up with the pricing changes, and it’s even more difficult to estimate how much your prompts and completions will cost until you see the bill. Tokencost works by counting the number of tokens in prompt and completion messages and multiplying that number by the corresponding model cost. Under the hood, it’s really just a simple cost dictionary and some utility functions for getting the prices right. It also accounts for…
- CACodeBurn – Analyze Claude Code token usage by taskApr 2026 · github.com · ▲112
Built this after realizing I was spending ~$1400/week on Claude Code with almost no visibility into what was actually consuming tokens. Tools like ccusage give a cost breakdown per model and per day, but I wanted to understand usage at the task level. CodeBurn reads the JSONL session transcripts that Claude Code stores locally (~/.claude/projects/) and classifies each turn into 13 categories based on tool usage patterns (no LLM calls involved). One surprising result: about 56% of my spend was on conversation turns with no tool usage. Actual coding (edits/writes) was…
- FFFrugon – Find which LLM calls a cheaper model could handle (local, MIT)Jul 2026 · github.com · ▲67
I started leaning in on AI heavily this year, as I wanted to get more done autonomously, but then my token usage climbed dramatically to the point where my weekly quota would run out before the end of the week, sometimes a couple of days into the week. I realised I had to do something about it else I'd have to double my spend. So I decided to start tracking my cost per task type. This revealed that a lot of my spend went to searches/scans or simple things like scouting tasks. I then decided to turn this into a simple CLI tool that can be used to read your OpenAI-style logs locally, and…
Geekflare Scraping API v2Apr 2026 · geekflare.com · ▲84RAG-ready web scraping that cuts your LLM token costs
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
