Plano – Edge and service proxy with orchestration for AI agents
Hey HN — I’m Adil from Katanemo (with Salman, Shuguang, and Meiyu) We previously shared an early version of this project as ArchGW. Based on customer feedback, the scope expanded from “LLM routing and model access” into something broader: delivery infrastructure for agentic applications. We renamed it to Plano and reworked the architecture accordingly. The problem On-the-ground AI practitioners will tell you that calling an LLM is not the hard part. The really hard part is delivering agentic applications to production quickly and reliably, then iterating without rewriting system code every…
What it does
In the maker’s words, at launch
Hey HN — I’m Adil from Katanemo (with Salman, Shuguang, and Meiyu) We previously shared an early version of this project as ArchGW. Based on customer feedback, the scope expanded from “LLM routing and model access” into something broader: delivery infrastructure for agentic applications. We renamed it to Plano and reworked the architecture accordingly. The problem On-the-ground AI practitioners will tell you that calling an LLM is not the hard part. The really hard part is delivering agentic applications to production quickly and reliably, then iterating without rewriting system code every time. In practice, teams keep rebuilding the same concerns that sit outside any single agent’s core logic: This includes model agility — the ability to pull from a large set of LLMs and swap providers without refactoring prompts or streaming handlers. They need to learn from production by collecting signals and traces that tell them what to fix. They need consistent policy enforcement for moderation and jailbreak protection, rather than sprinkling hooks across codebases. And they need multi-agent patterns like handoff and specialization without turning their app into orchestration glue. These concerns get rebuilt and maintained inside fast-changing frameworks and application code, coupling product logic to infrastructure decisions. It’s brittle, and pulls teams away from core product work into plumbing they shouldn’t have to own. What Plano does Plano moves core delivery concerns out of process into a modular proxy and dataplane designed for agents. It supports inbound listeners (agent orchestration, safety and moderation hooks), outbound listeners (hosted or API-based LLM routing), or both together. Plano provides the following capabilities via a unified, protocol-native, framework-friendly dataplane: - Orchestration: Low-latency routing and handoff between agents. Add or change agents without modifying app code, and evolve strategies centrally instead of duplicating logic across services. - Guardrails & Memory Hooks: Apply jailbreak protection, content policies, and context workflows (rewriting, retrieval, redaction) once via filter chains. This centralizes governance and ensures consistent behavior across your stack. - Model Agility: Route by model name, semantic alias, or preference-based policies. Swap or add models without refactoring prompts, tool calls, or streaming handlers. - Agentic Signals™: Zero-code capture of behavior signals, traces, and metrics across every agent, surfacing traces, token usage, and learning signals in one place. The goal is to keep application code focused on product logic while Plano owns delivery mechanics. More on Architecture Plano has two main parts: Envoy-based data plane. Uses Envoy’s HTTP connection management to talk to model APIs, services, and tool backends. We didn’t build a separate model server—Envoy already handles streaming, retries, timeouts, and connection pooling. Some of us are core Envoy contributors at Katanemo. Brightstaff, a lightweight controller written in Rust. It inspects prompts and conversation state, decides which upstreams to call and in what order, and coordinates routing and fallback. It uses small LLMs (1–4B parameters) trained for constrained routing and orchestration. These models do not generate responses and fall back to static policies on failure. The models are open sourced here: https://huggingface.co/katanemo Plano runs alongside your app servers (cloud, on-prem, or local dev), doesn’t require a GPU, and leaves GPUs where your models are hosted. Repo https://github.com/katanemo/plano + docs https://docs.planoai.dev/
Does a similar job
all alternatives →
- AAArchGW – An intelligent edge and service proxy for agents2025 · github.com · ▲118
Hey HN! This is Adil, Salman and Jose and and we’re behind archgw [1]. An intelligent proxy server designed as an edge and AI gateway for agents - one that natively know how to handle prompts, not just network traffic. We’ve made several sweeping changes so sharing the project again. A bit of background on why we’ve built this project. Building AI agent demos is easy, but to create something production-ready there is a lot of repeat low-level plumbing work that everyone is doing. You’re applying guardrails to make sure unsafe or off-topic requests don’t get through. You’re clarifying vague…
- ARArch-Router – 1.5B model for LLM routing by preferences, not benchmarks2025 · ▲66
Hi HN — we're the team behind Arch (https://github.com/katanemo/archgw), an open-source proxy for LLMs written in Rust. Today we're releasing Arch-Router (https://huggingface.co/katanemo/Arch-Router-1.5B), a 1.5B router model for preference-based routing, now integrated into the proxy. As teams integrate multiple LLMs - each with different strengths, styles, or cost/latency profiles — routing the right prompt to the right model becomes a critical part of the application design. But it's still an open problem. Most routing systems fall into two…
- AOarchgw: open-source, intelligent proxy for AI agents, built on Envoy2024 · github.com · ▲20
Hi HN! This is Adil, Salman, Co and Shuguang and we're excited to introduce archgw [1], an open source intelligent proxy for agents built on Envoy [2]. Arch moves the critical but crufty work around safety, observability, and routing of prompts outside business logic. Arch is a uniquely intelligent infrastructure primitive, engineered with purpose-built fast LLMs [3] for tasks like intent detection over multi-turn, parameter identification and extraction, triggering single/multiple function calls, and offers convenience features to auto dispatch LLM calls for summarization based on data…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 19d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, January 2026
the whole month →- IN
Hey HN! I wanted to share something I built over the last few weeks: isometric.nyc is a massive isometric pixel art map of NYC, built with nano banana and coding agents. I didn't write a single line of code. Of course no-code doesn't mean no-engineering. This project took a lot more manual labor than I'd hoped! I wrote a deep dive on the workflow and some thoughts about the future of AI coding and creativity: http://cannoneyed.com/projects/isometric-nyc
AI · Jan 2026 · cannoneyed.com
Automatic AI-powered code reviews the moment you open a PR
Dev tools · Jan 2026 · kilo.ai
