An open-source AI Gateway with integrated guardrails
Hi HN, I've been developing Portkey Gateway, an open-source AI gateway that's now processing billions of tokens daily across 200+ LLMs. Today, we're launching a significant update: integrated Guardrails at the gateway level. Key technical features: 1. Guardrails as middleware: We've implemented a hooks architecture that allows guardrails to act as middleware in the request/response flow. This enables real-time LLM output evaluation and transformation. 2. Flexible orchestration: The gateway can now route requests based on guardrail verdicts. This allows for complex logic like fallbacks…
An open-source AI Gateway with integrated guardrails launched on August 14, 2024 with 21 votes, #973 of 5,471 launches that month and more than 75% of that year's launches. The site was still online in September 2026, 2 years on.
In plain words
Portkey Gateway is an open-source AI gateway that routes requests across 200+ large language models while processing billions of tokens daily. Its latest update adds integrated guardrails that operate as middleware to evaluate and transform LLM outputs in real time. The system features flexible request routing based on guardrail verdicts, enabling fallbacks and conditional logic, plus a modular plugin system for various guardrail implementations. It is designed for developers and organizations managing multiple LLM integrations who need output validation and quality control at the gateway level.
written from the facts on this page · September 2026
From the sources
A blazing fast AI Gateway with integrated guardrails. Route to 1,600+ LLMs, 50+ AI Guardrails with 1 fast & friendly API. - Guardrails on the Gateway Framework · Portkey-AI/gateway Wiki
About a year ago, we started Portkey to solve a problem we'd faced ourselves — lack of adequate tooling to confidently put LLM apps in production. This showed up in many ways: We started building Portkey to solve these "ops" challenges, and we did it in our own way, with our opinionated, open source AI Gateway. (that reminds - have you starred this repo? 😬) With Portkey Gateway, we now process billions of LLM tokens every day, and help hundreds of companies take their AI apps to production. Teams use Portkey to debug fast, iterate fast, handle API failures, and much more! We built the Gateway to avoid affecting core LLM behavior. Portkey performs request/response transformations across…from github.com
In the maker’s words, at launch
Hi HN, I've been developing Portkey Gateway, an open-source AI gateway that's now processing billions of tokens daily across 200+ LLMs. Today, we're launching a significant update: integrated Guardrails at the gateway level. Key technical features: 1. Guardrails as middleware: We've implemented a hooks architecture…
Does a similar job
all alternatives →
TrueFoundry AI GatewayDec 2025 · truefoundry.com · ▲315Many teams connecting LLMs to external tools eventually encounter the same architectural issue: as more tools and agents are added, the integration pattern becomes an N×M mesh of direct connections. Each agent implements its own auth, retries, rate limiting, and logging; each tool needs credentials distributed to multiple places and observability becomes fragmented. We built LLM gateway with this goal to provide a single place to manage authentication, authorization, routing, and observability for MCP servers, with a path toward a more general agent-gateway architecture in the future. The…
LLM Gateway2025 · llmgateway.io · ▲296I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…
A lightweight AI gateway to 100+ models, in TS2024 · github.com · ▲37We are Rohit & Ayush, we created Portkey this year March to help tackle some challenges we had seen while building apps based on GPT3, 3.5, 4, and the DevOps principles we brought to the scene to help tackle them. We believe, a solid, performant, and reliable gateway lays the foundation to help build the next level of LLM apps. It decreases excessive reliance on any one company and takes the focus back to building instead of spending time fixing the nitty gritties of different providers and making them work together. Features: Blazing fast (9.9x faster) with a tiny footprint (~45kb…
Forge – Guardrails take an 8B model from 53% to 99% on agentic tasksMay 2026 · github.com · ▲687Hi HN, I'm Antoine Zambelli, AI Director at Texas Instruments. I built Forge, an open-source reliability layer for self-hosted LLM tool-calling. What it does: - Adds domain-and-tool-agnostic guardrails (retry nudges, step enforcement, error recovery, VRAM-aware context management) to local models running on consumer hardware - Takes an 8B model from ~53% to ~99% on multi-step agentic workflows without changing the model - just the system around it - Ships with an eval harness and interactive dashboard so you can reproduce every number I wanted to run a handful of always-on agentic systems…
FastRouter.ai22d ago · fastrouter.ai · ▲364Route requests to the right LLM for cost, latency & quality
Context GatewayMar 2026 · github.com · ▲220We built an open-source proxy that sits between coding agents (Claude Code, OpenClaw, etc.) and the LLM, compressing tool outputs before they enter the context window. Demo: https://www.youtube.com/watch?v=-vFZ6MPrwjw#t=9s. Motivation: Agents are terrible at managing context. A single file read or grep can dump thousands of tokens into the window, most of it noise. This isn't just expensive — it actively degrades quality. Long-context benchmarks consistently show steep accuracy drops as context grows (OpenAI's GPT-5.4 eval goes from 97.2% at 32k to 36.6% at 1M…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, August 2024
the whole month →
Jupitrr AI▲1,195Jupitrr AI turns raw talking videos into scroll-stopping content for coaches and consultants. Research, script, record, edit, and publish in one app.
AI · 2024 · jupitrr.com
Hey there HN! We’re Joe and Stopa, and today we’re open sourcing InstantDB, a client-side database that makes it easy to build real-time and collaborative apps like Notion and Figma. Building modern apps these days involves a lot of schleps. For a basic CRUD app you need to spin up servers, wire up endpoints, integrate auth, add permissions, and then marshal data from the backend to the frontend and back again. If you want to deliver a buttery smooth user experience, you’ll need to add optimistic updates and rollbacks. We do these steps over and over for every feature we build, which can…
Dev tools · 2024 · github.com