Millwright – Rust-based, self-hosted LLM router
Hey HN, With the news of OpenRouter possibly being acquired and proliferation of hosted LLM routers (i.e. Ramp Router, Vercel’s AI Gateway), I saw the need for a self hosted solution focused on cost savings, transparency, and performance. So, I built an open sourced router with a simple CLI interface that can easily sit between coding agents and GenAI workloads. For the curious and lazy, at the moment, Millwright has the tools for, - Providers: OpenAI-compatible APIs, Anthropic, Amazon Bedrock - Routing: policy-controlled model roles (cheap, mid, frontier), cheapest healthy route selection -…
Millwright – Rust-based, self-hosted LLM router launched on July 22, 2026 with 10 votes, #1,657 of 24,623 launches that month and more than 90% of that year's launches.
In plain words
Millwright is a self-hosted, open-source LLM router built in Rust that sits between applications and multiple AI providers to optimize costs and performance. It supports OpenAI-compatible APIs, Anthropic, and Amazon Bedrock, with policy-based routing that automatically selects the cheapest healthy model option across cost tiers. Designed for developers and teams running coding agents and generative AI workloads, it offers spend tracking, request tracing, and cache management through a simple CLI interface, prioritizing transparency and cost savings over hosted alternatives.
written from the facts on this page · September 2026
From the sources
Self-hosted LLM router. Cost effective, deterministic, and fast. Secure and private by default. - Northwood-Systems/millwright
In the maker’s words, at launch
Hey HN, With the news of OpenRouter possibly being acquired and proliferation of hosted LLM routers (i.e. Ramp Router, Vercel’s AI Gateway), I saw the need for a self hosted solution focused on cost savings, transparency, and performance. So, I built an open sourced router with a simple CLI interface that can easily…
Does a similar job
all alternatives →
Foreman, a self-hosted LLM gateway for cost aware model routingJul 2026 · github.com · ▲15Self-hosted LLM router. Cost effective, deterministic, and fast. Secure and private by default. - Northwood-Systems/millwright
LLM Gateway2025 · llmgateway.io · ▲296I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…
Arch-Router – 1.5B model for LLM routing by preferences, not benchmarks2025 · ▲66Hi HN — we're the team behind Arch (https://github.com/katanemo/archgw), an open-source proxy for LLMs written in Rust. Today we're releasing Arch-Router (https://huggingface.co/katanemo/Arch-Router-1.5B), a 1.5B router model for preference-based routing, now integrated into the proxy. As teams integrate multiple LLMs - each with different strengths, styles, or cost/latency profiles — routing the right prompt to the right model becomes a critical part of the application design. But it's still an open problem. Most routing systems fall into two…
Weave Router 2.0Sep 2026 · weaveos.com · ▲341Subscription Aware Coding Agent Router (GPT-6 Intelligence)
Smart model routing directly in Claude, Codex and CursorJun 2026 · github.com · ▲216We built a model router that plugs into coding agents (e.g. Claude Code, Codex, Cursor, etc.) and intelligently sends requests to the best model to serve them. Here's a quick demo of running it locally: https://www.youtube.com/watch?v=isKhAyivtfM. At Weave, we write most of our code with AI, and it's been getting more expensive. This came to a head when Opus 4.7 was released and, thanks to its tokenizer changes, our costs shot up. We knew we didn't need Opus for everything but we didn't want to lose out on the intelligence for the cases where you really need it. So we decided…
FastRouter.ai21d ago · fastrouter.ai · ▲364Route requests to the right LLM for cost, latency & quality
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, July 2026
the whole month →- IR
I might be the only SRE on Earth with his own bowling center. It's a more in-depth gig than you'd think. My family and I bought an abandoned 8-lane bowling center in the rural mid-west. In our small town there weren't many recreation options for families. You've heard of a food desert? This is an R&R desert. It had been abandoned for a good reason. The roof leaks, the electrical system was constantly surging, and my 70-year-old bowling equipment (still) doesn't work perfectly. The system that keeps your score is particularly interesting to me. It's the thing you watch during your game, but…
Life & fun · Jul 2026
18 Words▲1,160Play 18 Words, the free daily word puzzle. The official home is 18words.com. Tap Play now to open today
Life & fun · Jul 2026 · 18words.com
- BA
Over the past few months, our team has been building more and more slidedecks using web frontend technologies with coding harnesses like Claude Code, but a common complaint is to make even small edits we need to edit the code either manually or via the harness. To avoid this loop, I ended up creating Bento, a single HTML file with everything you need in a slide tool including animations and shared editing. There's no install or cloud login, everything works offline. The default deck is around 560 KB and it doesn't need to fetch anything once you got it. Open it in a browser and then you can…
Dev tools · Jul 2026 · bento.page
A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal computer like mine," and above all, "I wonder if it would work without going into OOM on a computer like mine." So I started working with the help of agents to test this possibility. I started converting the model to int4, understanding MTP usage, and if possible implementing DSA for long context.…
AI · Jul 2026 · github.com
OpenSEO▲928Hello! For the last 2 months, I've been building OpenSEO. It's a user friendly UI wrapping the DataForSEO API which has high quality SEO data for keywords, backlinks, AI SEO and competitor research. Instead of paying $100+ per month, you buy DataForSEO credits and pay only for usage. The coolest thing about the project so far has been hearing about what people are building on top of it. People are using coding agents to quickly integrate with Google Search Console, Google Ads and building their own AI SEO and reporting features. Many were already building their own custom solutions around…
Dev tools · Jul 2026 · openseo.so