Prompt-refiner – Lightweight optimization for LLM inputs and RAG
Hi HN, While building RAG agents, I noticed a lot of token budget was wasted on formatting overhead (HTML tags, JSON structure, whitespace). Existing solutions felt too heavy (often requiring torch/transformers), so I wrote this lightweight, zero-dependency library to solve it. It includes strategies for context packing, PII redaction, and tool output compression. Benchmarks show it can save ~15% of tokens with negligible latency overhead (<0.5ms). Happy to answer any questions!
Does the same job
all alternatives →
- IBI Built a UI to finetune LLMs x100 faster2024 · finetuna-ui.com · ▲7
After fine-tuning GPT for a personal project, I realized how tedious it is to write plain text in a massive JSON file. That's why I built this app for my own use, and I want to see if others could benefit from a tool like this as well ;)
- CSCtx, save tokens by loading only the relevant toolsJun 2026 · github.com · ▲8
Hi HN! Token cost has started to become a high topic of concern to all of us. I tried a few (awesome) tools such as rtk, caveman, and the recent (hillarious but effective) ponytail. What they usually do, is in-line token reduction, e.g. try to compress requests / responses as much as possible. But then it hit me (and I’m sure others had similar ideas) - just like we have routers that pick the right model, why not have something that will also narrow down the amount of available tools, skills and mcps based on repo/context? People usually accumulate skills, agents, MCP servers,…
- AEAn extensive set of RAG implementations+many different strategies2024 · github.com · ▲6
Hi all, Sharing a repo I was working on for a while. It’s open-source and includes many different strategies for RAG (currently 17), including tutorials, and visualizations. This is great learning and reference material. Open issues, suggest more strategies, and use as needed. Enjoy!
- ILImprove LLM Performance by Maximizing Iteration Speed2024 · palico.ai · ▲5
LLM Application development is extremely iterative, more so than any other types of development. This is because in addition to all the activities involved in regular application development, we also need to make the LLM Application accurate and reduce hallucination. To improve performance, we need to trial and error various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, try different agent architecture, and more. There are thousands of permutations to try. We need to be able to easily experiment with these…
- A1A 100-Line LLM Framework2025 · github.com · ▲9
I've seen a lot of comments about how complex frameworks like LangChain can be. Over the holidays, I wanted to see how minimal an LLM framework could get if we stripped away everything non-essential. The result is an LLM framework in just 100 lines of code. These 100 lines capture what I see as the core abstraction of most LLM frameworks: a nested directed graph that breaks down tasks into multiple LLM steps, with branching and recursion to enable agent-like decision-making. From there, you can layer on more advanced features like agents, RAG, task decomposition, and more. I’ve intentionally…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, December 2025
the whole month →- GP
Life & fun · Dec 2025 · dosaygo-studio.github.io
- JG
Hi everyone! My name's Luke and I made the original Jmail here alongside Riley Walz. We had a ton of friends collaborate on building out more of the app suite last night in lieue of DOJ's "Epstein files" release. Please AMA!
Life & fun · Dec 2025 · jmail.world
PlanEat AI▲733AI turns your health goals into a 7-day menu & grocery list
AI · Dec 2025 · planeatai.com
- 2G
Community, All the HN belong to you. This is an archive of hacker news that fits in your browser. When I made HN Made of Primes I realized I could probably do this offline sqlite/wasm thing with the whole GBs of archive. The whole dataset. So I tried it, and this is it. Have Hacker News on your device. Go to this repo (https://github.com/DOSAYGO-STUDIO/HackerBook): you can download it. Big Query -> ETL -> npx serve docs - that's it. 20 years of HN arguments and beauty, can be yours forever. So they'll never die. Ever. It's the unkillable static archive of HN and it's…
Dev tools · Dec 2025 · hackerbook.dosaygo.com
