AI · alternatives · 2026
24 alternatives to Mantis, A self-hosted LLM gateway
Hey HNers - Riz here. I got together with a few guys and we built an LLM gateway. It's designed for small teams working on early-stage products, and can be deployed to AWS using a single command (i.e. `mantis deploy`). It's self-hosted, and is designed to belong to you.
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.
- 1
Many teams connecting LLMs to external tools eventually encounter the same architectural issue: as more tools and agents are added, the integration pattern becomes an N×M mesh of direct connections. Each agent implements its own auth, retries, rate limiting, and logging; each tool needs credentials distributed to multiple places and observability becomes fragmented. We built LLM gateway with this goal to provide a single place to manage authentication, authorization, routing, and observability for MCP servers, with a path toward a more general agent-gateway architecture in the future. The…
Dec 2025 · truefoundry.com · its alternatives →
- 2
Scoutflo Deploy▲501Automate self-hosting of commercial open source products
2023 · its alternatives →
- 3
Arch▲314Hey HN — I’m Adil from Katanemo (with Salman, Shuguang, and Meiyu) We previously shared an early version of this project as ArchGW. Based on customer feedback, the scope expanded from “LLM routing and model access” into something broader: delivery infrastructure for agentic applications. We renamed it to Plano and reworked the architecture accordingly. The problem On-the-ground AI practitioners will tell you that calling an LLM is not the hard part. The really hard part is delivering agentic applications to production quickly and reliably, then iterating without rewriting system code every…
2024 · github.com · its alternatives →
- 4
LLM Gatewayfree tier▲296I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…
2025 · llmgateway.io · its alternatives →
- 5
ZenMux▲382An enterprise-grade LLM gateway with automatic compensation
Feb 2026 · zenmux.ai · its alternatives →
- 6
an agentic cms self-hosted on cloudflare, for podcasts, blogs, photos, videos, documents, and curated urls. - microfeed/microfeed
2022 · github.com · its alternatives →
- 7
- 8
Your very own offline-first privacy-centric open-source data-center! https://homelabos.com/
2018 · gitlab.com · its alternatives →
- 9
We built any-llm because we needed a lightweight router for LLM providers with minimal overhead. Switching between models is just a string change : update "openai/gpt-4" to "anthropic/claude-3" and you're done. It uses official provider SDKs when available, which helps since providers handle their own compatibility updates. No proxy or gateway service needed either, so getting started is pretty straightforward - just pip install and import. Currently supports 20+ providers including OpenAI, Anthropic, Google, Mistral, and AWS Bedrock. Would love to hear what you think!
2025 · github.com · its alternatives →
- 10
Self-hosted LLM router. Cost effective, deterministic, and fast. Secure and private by default. - Northwood-Systems/millwright
Jul 2026 · github.com · its alternatives →
- 11SH
2024 · openai.servicestack.net · its alternatives →
- 12AS
Hi HN, I'm Alon, and I'm building Alien, an open-source platform for deploying your software into your customer's environment and keeping it fully managed. In my previous startup, I heard the same question from every single enterprise customer over and over again: "My data is sensitive. Can I deploy your product to my own cloud account?" Self-hosting is becoming very popular because it lets users keep their data private, local, and inside their own environment. Unfortunately, self-hosting breaks down when someone starts paying for your software. Especially if it's an enterprise customer.…
Apr 2026 · its alternatives →
- 13
The open source LLM router that connects your AI agents and harnesses to any provider in seconds. Subscriptions, pay-per-token, local models, and custom providers.
2024 · manifest.build · its alternatives →
- 14AG
Hi HN My name is Salman and I work on Arch GW - the intelligent gateway designed to protect, observe, and personalize LLM applications with your APIs. https://github.com/katanemo/arch Our team built Envoy Proxy at Lyft, and re-imagined it with the belief that: Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems for personalization – all outside business logic. Engineered with purpose-built LLMs, Arch…
2024 · docs.archgw.com · its alternatives →
- 15
It's a self-hosted mailbox for all emails coming to your own domains, with Web interface and CLI included. Notes: - all emails share the same inbox, as long as set your DNS correctly - Web client via self-hosted Docker or Cloudflare Pages - need additional access controls, may use Basic Auth or Forward Auth with Cloudflare deployment
2024 · github.com · its alternatives →
- 16
Deploy LLMs to production in one command. VeloxML CLI is an open-source engine that provisions optimized serverless infrastructure directly in your own AWS/GCP account - paguasmar/veloxml-deploy
Sep 2026 · github.com · its alternatives →
- 17
ModelRiver▲37Connect once to every major AI model. Run AI workflows in the background while your backend controls what AI sees.
Jan 2026 · modelriver.com · its alternatives →
- 18
Contribute to fmeringdal/do-hashicorp-cluster development by creating an account on GitHub.
2021 · github.com · its alternatives →
- 19
Talk to multiple LLMs thru a unified API
2025 · ai-gateway.dev · its alternatives →
- 20
A zero-trust security layer between your apps and LLMs
Jan 2026 · its alternatives →
- 21
We’ve been building local and on-prem agent workflows for open-source LLMs. Engines like Ollama or vLLM ship with no auth, so every team ends up writing the same JWT proxy. Attach Gateway is a single process that sits in front of any model server and handles the boring bits: - verifies OIDC / DID JWTs - adds X-Attach-User and X-Attach-Session headers so downstream agents share the same identity - optional /a2a/tasks/send endpoint for Google-style A2A hand-offs - mirrors prompts + completions to Weaviate (runs in Docker) One `pip install attach-dev`, export a token, run…
2025 · github.com · its alternatives →
- 22
Over the past couple weeks, l've been really immersed in learning about MCP, a new protocol for equipping any LLM with a set of tools that can run on your own machine or a remote server you control and give all kinds of superpowers to Al agents to do things like search, etc. As part of that research, l've already built one very fleshed-out and useful MCP server that l've shared here (I've added much more to it recently though!), the LLM Gateway MCP Server, which lets you use a big model to delegate to a cheaper model (and many more things in addition to that, like running automated…
2025 · github.com · its alternatives →
- 23OS
2018 · medium.com · its alternatives →
- 24MI
I've been working on a platform that uses LLMs to build maintain and manage k8s clusters on any cloud. The system writes infra as code to your Github repos and automatically containerizes and scales any services (public or private). The goal is to give your average engineer a vercel-like deployment experience for any service in any language at minimal cost. We have humans involved at the moment auditing LLM outputs and keeping an eye on clusters. We are looking for folks who may be thinking about their first infra/devops hire. Just connect your github and your cloud provider. The system…
2024 · milkinfrastructure.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →