Alternatives
Products that do what Mantis, A self-hosted LLM gateway does
Hey HNers - Riz here. I got together with a few guys and we built an LLM gateway. It's designed for small teams working on early-stage products, and can be deployed to AWS using a single command (i.e. `mantis deploy`). It's self-hosted, and is designed to belong to you.
- 1
- 2
- 3
- 4SH
2022 · github.com
- 5
- 6
- 7
- 8
- 9SL
2017 · servers.lol
- 10
- 11HA
2018 · gitlab.com
- 12AL
We built any-llm because we needed a lightweight router for LLM providers with minimal overhead. Switching between models is just a string change : update "openai/gpt-4" to "anthropic/claude-3" and you're done. It uses official provider SDKs when available, which helps since providers handle their own compatibility updates. No proxy or gateway service needed either, so getting started is pretty straightforward - just pip install and import. Currently supports 20+ providers including OpenAI, Anthropic, Google, Mistral, and AWS Bedrock. Would love to hear what you think!
2025 · github.com
- 13SH
2024 · openai.servicestack.net
- 14MG
Many teams connecting LLMs to external tools eventually encounter the same architectural issue: as more tools and agents are added, the integration pattern becomes an N×M mesh of direct connections. Each agent implements its own auth, retries, rate limiting, and logging; each tool needs credentials distributed to multiple places and observability becomes fragmented. We built LLM gateway with this goal to provide a single place to manage authentication, authorization, routing, and observability for MCP servers, with a path toward a more general agent-gateway architecture in the future. The…
Dec 2025 · truefoundry.com
- 15AS
Hi HN, I'm Alon, and I'm building Alien, an open-source platform for deploying your software into your customer's environment and keeping it fully managed. In my previous startup, I heard the same question from every single enterprise customer over and over again: "My data is sensitive. Can I deploy your product to my own cloud account?" Self-hosting is becoming very popular because it lets users keep their data private, local, and inside their own environment. Unfortunately, self-hosting breaks down when someone starts paying for your software. Especially if it's an enterprise customer.…
Apr 2026
- 16FA
Jul 2026 · github.com
- 17AG
Hi HN My name is Salman and I work on Arch GW - the intelligent gateway designed to protect, observe, and personalize LLM applications with your APIs. https://github.com/katanemo/arch Our team built Envoy Proxy at Lyft, and re-imagined it with the belief that: Prompts are nuanced and opaque user requests, which require the same capabilities as traditional HTTP requests including secure handling, intelligent routing, robust observability, and integration with backend (API) systems for personalization – all outside business logic. Engineered with purpose-built LLMs, Arch…
2024 · docs.archgw.com
- 18AR
Hi HN — we're the team behind Arch (https://github.com/katanemo/archgw), an open-source proxy for LLMs written in Rust. Today we're releasing Arch-Router (https://huggingface.co/katanemo/Arch-Router-1.5B), a 1.5B router model for preference-based routing, now integrated into the proxy. As teams integrate multiple LLMs - each with different strengths, styles, or cost/latency profiles — routing the right prompt to the right model becomes a critical part of the application design. But it's still an open problem. Most routing systems fall into two…
2025
- 19VE
It's a self-hosted mailbox for all emails coming to your own domains, with Web interface and CLI included. Notes: - all emails share the same inbox, as long as set your DNS correctly - Web client via self-hosted Docker or Cloudflare Pages - need additional access controls, may use Basic Auth or Forward Auth with Cloudflare deployment
2024 · github.com
- 20AG
We’ve been building local and on-prem agent workflows for open-source LLMs. Engines like Ollama or vLLM ship with no auth, so every team ends up writing the same JWT proxy. Attach Gateway is a single process that sits in front of any model server and handles the boring bits: - verifies OIDC / DID JWTs - adds X-Attach-User and X-Attach-Session headers so downstream agents share the same identity - optional /a2a/tasks/send endpoint for Google-style A2A hand-offs - mirrors prompts + completions to Weaviate (runs in Docker) One `pip install attach-dev`, export a token, run…
2025 · github.com
- 21TU
Over the past couple weeks, l've been really immersed in learning about MCP, a new protocol for equipping any LLM with a set of tools that can run on your own machine or a remote server you control and give all kinds of superpowers to Al agents to do things like search, etc. As part of that research, l've already built one very fleshed-out and useful MCP server that l've shared here (I've added much more to it recently though!), the LLM Gateway MCP Server, which lets you use a big model to delegate to a cheaper model (and many more things in addition to that, like running automated…
2025 · github.com
- 22MI
I've been working on a platform that uses LLMs to build maintain and manage k8s clusters on any cloud. The system writes infra as code to your Github repos and automatically containerizes and scales any services (public or private). The goal is to give your average engineer a vercel-like deployment experience for any service in any language at minimal cost. We have humans involved at the moment auditing LLM outputs and keeping an eye on clusters. We are looking for folks who may be thinking about their first infra/devops hire. Just connect your github and your cloud provider. The system…
2024 · milkinfrastructure.com
- 23
Contain LiteLLM, Bifrost, Portkey.. behind one control plane
14d ago · agnos-llm-gateway.site
- 24MI
2021 · mantis.chat
Ranked by how close each launch is in meaning, then by votes. Refine with a description →