Dev tools · alternatives · 2026
24 alternatives to Self-host open-source LLMs on AWS with scale-to-zero
Deploy LLMs to production in one command. VeloxML CLI is an open-source engine that provisions optimized serverless infrastructure directly in your own AWS/GCP account - paguasmar/veloxml-deploy
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1ML
2025 · simonwillison.net · its alternatives →
- 2TC
2020 · npmjs.com · its alternatives →
- 3AC
2024 · github.com · its alternatives →
- 4
LLMWare▲358Dev tool to make AI apps to deploy privately or locally
2024 · llmware.ai · its alternatives →
- 5
AskCodi▲230Custom LLMs, without training. Use via openai compatible api
Nov 2025 · askcodi.com · its alternatives →
- 6PD
We’re Robin, Louis, and Thomas. Pipelex is a DSL and a Python runtime for repeatable AI workflows. Think Dockerfile/SQL for multi-step LLM pipelines: you declare steps and interfaces; any model/provider can fill them. Why this instead of yet another workflow builder? - Declarative, not glue code: you state what to do; the runtime figures out how. - Agent-first: each step carries natural-language context (purpose, inputs/outputs with meaning) so LLMs can follow, audit, and optimize. Our MCP server enables agents to run pipelines but also to build new pipelines on demand. - Open…
Oct 2025 · github.com · its alternatives →
- 7LT
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com · its alternatives →
- 8LA
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
- 9
- 10OC
Hey HN, I’ve built Open Codex, a fully local, open-source alternative to OpenAI’s Codex CLI. My initial plan was to fork their project and extend it. I even started doing that. But it turned out their code has several leaky abstractions, which made it hard to override core behavior cleanly. Shortly after, OpenAI introduced breaking changes. Maintaining my customizations on top became increasingly difficult. So I rewrote the whole thing from scratch using Python. My version is designed to support local LLMs. Right now, it only works with phi-4-mini (GGUF) via…
2025 · github.com · its alternatives →
- 11
Every MCP server injects its full tool schemas into context on every turn — 30 tools costs ~3,600 tokens/turn whether the model uses them or not. Over 25 turns with 120 tools, that's 362,000 tokens just for schemas. mcp2cli turns any MCP server or OpenAPI spec into a CLI at runtime. The LLM discovers tools on demand: mcp2cli --mcp https://mcp.example.com/sse --list # ~16 tokens/tool mcp2cli --mcp https://mcp.example.com/sse create-task --help # ~120 tokens, once mcp2cli --mcp https://mcp.example.com/sse create-task --title "Fix bug" No…
Mar 2026 · github.com · its alternatives →
- 12LT
This is my take on the common "use llms to generate shell commands" utility. Emphasis is placed on good CLI UX, simplicity, and flexibility. `llm2sh` supports multiple LLM providers and lets LLMs generate multi-command sequences to handle complex tasks. There is also limited support for commands requiring `sudo` and other basic input. I recommend using Groq llama3-70b for day-to-day use. The ultra-low latency is a game-changer - its near-instant responses helps `llm2sh` integrate seamlessly into day-to-day tasks without breaking you out of the 'zone'. For more advanced tasks, swapping to…
2024 · github.com · its alternatives →
- 13YA
Built this for my LLM workflows - needed searchable, persistent memory that wouldn't blow up storage costs. I also wanted to use it locally for my research. It's a content-addressed storage system with block-level deduplication (saves 30-40% on typical codebases). I have integrated the CLI tool into most of my workflows in Zed, Claude Code, and Cursor, and I provide the prompt I'm currently using in the repo. The project is in C++ and the build system is rough around the edges but is tested on macOS and Ubuntu 24.04.
2025 · github.com · its alternatives →
- 14AC
2024 · github.com · its alternatives →
- 15TO
Hey HN – We’re Danny and Ethan from Telophase (https://telophase.dev). We’re building automation to isolate cloud environments by tenant or team. You can think of us as an Open-Source AWS Control Tower[1]. Our CLI is a drop-in tool to manage multi-account AWS setups. To start, we support AWS Account Factory and landing zones defined in AWS CDK. We plan to support multi-project GCP and multi-subscription Azure setups soon! Features: 1. Manage accounts in code. 2. Provision accounts and their infrastructure with a single command, `telophase deploy`. 3. Deploy to a subset of accounts…
2023 · github.com · its alternatives →
- 16
LLM-CLI▲13Cloud and local LLM AI assistant for the command line
2025 · github.com · its alternatives →
- 17
Bash4LLM is a single-file Bash wrapper for interacting with LLMs from the terminal. I created it because I wanted something simple that worked without installing Python, Node, or any other runtime. It uses only Bash, curl, and jq. You can send prompts, start a small chat, process files line by line, stream output, and save session metadata in JSON format. I tried to make it safe and predictable: no use of the system /tmp, no use of eval. Groq is supported by default, and other providers can be added with dedicated Bash scripts in the extras/providers/ folder. Example: echo…
Jun 2026 · github.com · its alternatives →
- 18OS
Hi everyone, we’re a small team, supported by Mozilla, who are working on re-imagining a UI for training, tuning and testing local LLMs. Everything is open source. If you’ve been training your own LLMs or have always wanted to, we’d love for you to play with the tool and give feedback on what the future development experience for LLM engineering could look like.
2025 · github.com · its alternatives →
- 19
Hi HN, We built LUML (https://github.com/luml-ai/luml), an open-source (Apache 2.0) MLOps/LLMOps platform that covers experiments, registry, LLM tracing, deployments and so on. It separates the control plane from your data and compute. Artifacts are self-contained. Each model artifact includes all metadata (including the experiment snapshots, dependencies, etc.), and it stays in your storage (S3-compatible or Azure). File transfers go directly between your machine and storage, and execution happens on compute nodes you host and connect to LUML. We’d love you to try…
Feb 2026 · github.com · its alternatives →
- 20TC
Hi HN, I built a CLI for uploading documents and querying them with an LLM agent that uses search tools rather than stuffing everything into the context window. I recorded a demo using the CrossFit 2025 rulebook that shows how this approach compares to traditional RAG and direct context injection[1]. The core insight is that LLMs running in loops with tool access are unreasonably effective at this kind of knowledge retrieval task[2]. Instead of hoping the right chunks make it into your context, the agent can iteratively search, refine queries, and reason about what it finds. The CLI handles…
2025 · github.com · its alternatives →
- 21
OrchestraML▲82From English prompt to deployed ML model with human approval
Jun 2026 · orchestra-ml.vercel.app · its alternatives →
- 22
I work as a SAP Integration consultant and built this as a side project. Friction point: Most self hosted LLM observability tools require Postgres, Redis and non trivial infrastructure. Teams just want to see what their agents are actually doing in Production, that set up cost discorages adoption. Torrix runs as a single docker contained backed by SQLite. The full install is: curl -o docker-compose.yml https://raw.githubusercontent.com/torrix-ai/install/main/doc... docker compose up No external dependencies. All data stays in a local SQLite file on your machine.…
May 2026 · github.com · its alternatives →
- 23MG
Hello HN, I've been working on this project for a while, and it has been in an "open" beta for some time. I finally believe it's ready for its first release. I hope you like it. Here are some potential questions that may arise: 1. How does it compare to LM Studio? It's likely that if you're already using LM Studio, you'll continue to do so. This project is designed to be more user-friendly. 2. Is it open-source? No, it is not. 3. Does it use any open-source libraries? Yes, it uses llama.cpp and a few others, as indicated in the license information included with the application. 4. Why is not…
2023 · avapls.com · its alternatives →
- 24AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
Also compare
- My LLM CLI tool can run tools now, from Python code or plugins alternatives
- Tiny CLI to save AWS costs in dev environments when you're sleeping alternatives
- A CLI tool I made to self-host any app with two commands on a VPS alternatives
- LLMWare alternatives
- AskCodi alternatives
- Pipelex – Declarative language for repeatable AI workflows alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →