Alternatives
Products that do what StreamMD does
Streaming MD for LLMs. 300x fewer chars parsed per token
- 1
RAG-ready web scraping that cuts your LLM token costs
Apr 2026 · geekflare.com
- 2TA
TokenDagger is a drop-in replacement for OpenAI’s Tiktoken (the tokenizer behind Llama 3, Mistral, GPT-3.*, etc.). It’s written in C++ 17 with thin Python bindings, keeps the exact same BPE vocab/special-token rules, and focuses on raw speed. I’m teaching myself LLM internals by re-implementing the stack from first principles. Profiling TikToken’s Python/Rust implementation showed a lot of time was spent doing regex matching. Most of my perf gains come from a) using a faster jit-compiled regex engine; and b) simplifying the algorithm to forego regex matching special tokens at all.…
2025 · github.com
- 3
- 4
- 5BT
2024 · github.com
- 6HW
TL;DR: Vector-based RAG performs poorly for many real-world applications like codebase chats, and you should consider 'language maps'. Part of our mission at Mutable.ai is to make it much easier for developers to build and understand software. One of the natural ways to do this is to create a codebase chat, that answer questions about your repo and help you build features. It might seem simple to plug in your codebase into a state-of-the-art LLM, but LLMs have two limitations that make human-level assistance with code difficult: 1. They currently have context windows that are too small to…
2024 · twitter.com
- 7
- 8IJ
Built this for streaming AI tool calls. LLMs stream function arguments as JSON character-by-character. Most parsers reparse from scratch each time - O(n²) behavior that causes UI lag. This maintains parsing state, processing only new characters. True O(n) performance that stays imperceptible throughout the entire response. Ruby gem, MIT licensed. Would love feedback.
Oct 2025 · aha.io
- 9RC
Hello HN! We're building a caching solution for LLMs (ChatGPT, Claude). By combining cutting-edge approaches, such as edge computing, prompt compression, vectorization, and others - it can reduce your AI bills by up to 10x and significantly lower response times. Key Features: - cost efficiency: our system stores frequent queries, reducing the number of upstream (paid) API calls - fast responses: with various nodes globally, we reduce latency by serving data from the nearest location - scalability: designed to handle increasing loads and data sizes without degrading performance. The cache…
2024 · edgematic.dev
- 10
- 11
- 12
- 13TF
I’d originally launched my app: Private LLM[1][2] on HN around 10 months ago, with a single RedPajama Chat 3B model. The app has come a long way since then. About a month ago, I added support for 4-bit OmniQuant quantized Mixtral 8x7B Instruct model, and it seems to outperform Q4 models at inference speed and Q8 models at text generation quality, while consuming only about 24GB of RAM[3] at 8k context length. The trick is: a) to use a better quantization algorithm and b) to use unquantized embeddings and the MoE gates (the overhead is quite small). Other notable features include many more…
2024
- 14LF
LLM from URL —— A free AI chat completion service directly from URL Usage: In the address bar of any web browser, type your question after https://818233.xyz/ and hit Enter to get the instant answer. You know the best part of this? Whitespace in the url is supported in most web browsers! You can also use curl or Wget to retrieve the appended url by replacing any whitespace with a '+' character. If you need to have an actual '+' character in your question, just use '++'. Example: The url "https://818233.xyz/hi there" in any web browser will return the same answer…
2025 · 818233.xyz
- 15
- 16LC
Hi HN, I'm building Librarian (https://uselibrarian.dev/), an open-source (MIT) context management tool that stops AI agents from burning tokens by blindly re-reading their entire conversation history on every turn. The Problem: If you're building agentic loops in frameworks like LangGraph or OpenClaw, you hit two walls fast: Financial Cost: Token usage scales quadratically over long conversations. Passing the whole history every time gets incredibly expensive. Context Rot: As the context window fills up, the LLM suffers from the "Lost in the Middle" effect. Response latency…
Feb 2026 · uselibrarian.dev
- 17
Reduce your LLM API bill 11–45% with zero code changes
Apr 2026 · textcompressor.unmutedlive.com
- 18NL
Refuel LLM (84.2%) outperforms trained human annotators (80.4%), GPT-3-5-turbo (81.3%), PaLM-2 (82.3%) and Claude (79.3%) across a benchmark of 15 text labeling datasets. It is a Llama-v2-13b base model, trained on over 2500 unique datasets (5.24B tokens) spanning categories such as classification, entity resolution, matching, reading comprehension and information extraction. Here is the interactive demo: https://labs.refuel.ai/playground. Pretty fun to play with!
2023
- 19AR
Hey HN, I wanted to share a UI toolkit project I’ve been working on recently, born out of how difficult I found it to build a great UX on top of LLMs, and keep application state in sync. I’ve built: - A React/JS front-end library for conversational interfaces, which makes it super easy to bootstrap AI assistants and ChatGPT style UX: https://github.com/nlkitai/nlux - A set of adapters that simplify integration with AI backends such as LangServe and HuggingFace The library is highly configurable, easy to theme, supports markdown streaming (that was tough to get…
2024 · github.com
- 20
Cut LLM costs. Free audit, pay only if it works.
Jun 2026 · decomp-ai.vercel.app
- 21AL
Raymond here from Butter.dev, an LLM response cache built as a chat-completions proxy. Today we're launching a key feature for the platform: the ability to generalize on dynamic, templated inputs. Caching at the HTTP request level has the obvious problem of generalizability. Nearly no request is identical, due to templated variables (like names) and metadata (like timestamps), so exact-match cache lookups rarely hit. We solve this at Butter by using LLMs to detect dynamic content in requests and derive their inter-relationships, allowing the cache entry to be stored as a template + variables…
Jan 2026 · blog.butter.dev
- 22
- 23CA
Current AI chat assistants face a fundamental challenge: context management in long conversations. While current LLM apps use multiple separate conversations to bypass context limits, a truly human-like AI assistant should maintain a single, coherent conversation thread, making efficient context management critical. Although modern LLMs have longer contexts, they still suffer from the long-context problem (e.g. context rot problem) - reasoning ability decreases as context grows longer. Memory-based systems have been invented to alleviate the context rot problem, however, memory-based…
Nov 2025
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →