Dev tools · alternatives · 2026
24 alternatives to Self-hosted gateway to access LLMs, Ollama, ComfyUI and FFmpeg servers
This self-hosted gateway allows developers to access multiple AI and media processing services through a unified interface, including LLMs, Ollama, ComfyUI, and FFmpeg servers. It enables local deployment and… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier. Self-hosted gateway to access LLMs, Ollama, ComfyUI and FFmpeg servers launched in 2024; newer entries below may have overtaken it.
- 1
LLM Gatewayfree tier▲296I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…
2025 · llmgateway.io · its alternatives →
- 2
Many teams connecting LLMs to external tools eventually encounter the same architectural issue: as more tools and agents are added, the integration pattern becomes an N×M mesh of direct connections. Each agent implements its own auth, retries, rate limiting, and logging; each tool needs credentials distributed to multiple places and observability becomes fragmented. We built LLM gateway with this goal to provide a single place to manage authentication, authorization, routing, and observability for MCP servers, with a path toward a more general agent-gateway architecture in the future. The…
Dec 2025 · truefoundry.com · its alternatives →
- 3
ZenMux▲382An enterprise-grade LLM gateway with automatic compensation
Feb 2026 · zenmux.ai · its alternatives →
- 4
Respan Gateway▲352One AI gateway with built-in observability and evals
Jun 2026 · respan.ai · its alternatives →
- 5
- 6
Scoutflo Deploy▲501Automate self-hosting of commercial open source products
2023 · its alternatives →
- 7
Context Gateway▲220We built an open-source proxy that sits between coding agents (Claude Code, OpenClaw, etc.) and the LLM, compressing tool outputs before they enter the context window. Demo: https://www.youtube.com/watch?v=-vFZ6MPrwjw#t=9s. Motivation: Agents are terrible at managing context. A single file read or grep can dump thousands of tokens into the window, most of it noise. This isn't just expensive — it actively degrades quality. Long-context benchmarks consistently show steep accuracy drops as context grows (OpenAI's GPT-5.4 eval goes from 97.2% at 32k to 36.6% at 1M…
Mar 2026 · github.com · its alternatives →
- 8
Use AI models without managing keys or billing
Dec 2025 · docs.netlify.com · its alternatives →
- 9
SUB/WAVE▲102Self-hosted radio with an AI DJ and one shared stream
Jul 2026 · getsubwave.com · its alternatives →
- 10
Talk to multiple LLMs thru a unified API
2025 · ai-gateway.dev · its alternatives →
- 11
- 12
We built any-llm because we needed a lightweight router for LLM providers with minimal overhead. Switching between models is just a string change : update "openai/gpt-4" to "anthropic/claude-3" and you're done. It uses official provider SDKs when available, which helps since providers handle their own compatibility updates. No proxy or gateway service needed either, so getting started is pretty straightforward - just pip install and import. Currently supports 20+ providers including OpenAI, Anthropic, Google, Mistral, and AWS Bedrock. Would love to hear what you think!
2025 · github.com · its alternatives →
- 13
SelfHostLLM▲134A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.
2025 · selfhostllm.org · its alternatives →
- 14
Proxy remote LLM API as Ollama and LM Studio, for using them in JetBrains AI Assistant - Stream29/ProxyAsLocalModel
2025 · github.com · its alternatives →
- 15
ModelRiver▲37Connect once to every major AI model. Run AI workflows in the background while your backend controls what AI sees.
Jan 2026 · modelriver.com · its alternatives →
- 16
One balance. Every model. Chat, image, video & audio.
Jun 2026 · lounge.llmgateway.io · its alternatives →
- 17
Self-hosted LLM router. Cost effective, deterministic, and fast. Secure and private by default. - Northwood-Systems/millwright
Jul 2026 · github.com · its alternatives →
- 18
Open-source, OpenAI-compatible LLM gateway you run yourself. One endpoint for 40+ providers, with virtual keys, budgets, and usage tracking. - mozilla-ai/otari
Jul 2026 · github.com · its alternatives →
- 19
[deprecated] AI Gateway - core infrastructure stack for building production-ready AI Applications - pyadav/gateway
2024 · github.com · its alternatives →
- 20
- 21
Hey HNers - Riz here. I got together with a few guys and we built an LLM gateway. It's designed for small teams working on early-stage products, and can be deployed to AWS using a single command (i.e. `mantis deploy`). It's self-hosted, and is designed to belong to you.
Jun 2026 · github.com · its alternatives →
- 22
Hi HN, I've been developing Portkey Gateway, an open-source AI gateway that's now processing billions of tokens daily across 200+ LLMs. Today, we're launching a significant update: integrated Guardrails at the gateway level. Key technical features: 1. Guardrails as middleware: We've implemented a hooks architecture that allows guardrails to act as middleware in the request/response flow. This enables real-time LLM output evaluation and transformation. 2. Flexible orchestration: The gateway can now route requests based on guardrail verdicts. This allows for complex logic like fallbacks…
2024 · github.com · its alternatives →
- 23
🦎 A lightweight, multi-protocol gateway for event-driven applications and AI agents. Expose and govern Kafka, MQTT, APIs, and MCP through one high-performance engine with shared routing, security, schema, and observability. - aklivity/zilla
2023 · github.com · its alternatives →
- 24
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →