Self-host open-source LLMs on AWS with scale-to-zero
Deploy LLMs to production in one command. VeloxML CLI is an open-source engine that provisions optimized serverless infrastructure directly in your own AWS/GCP account - paguasmar/veloxml-deploy
Self-host open-source LLMs on AWS with scale-to-zero launched on September 9, 2026 with 7 votes, #374 of 5,365 launches that month and more than 75% of that year's launches.
What it does
Deploy open-source LLMs directly to your own AWS/GCP account with one command. Zero Docker, zero Kubernetes, scale-to-zero. This repo is still under heavy development and the documentation is evolving. You're welcome to try it, but expect some breaking changes. Watch "releases" of this repo to receive a notification when we are ready for Beta. And give us a star if you like it! # 1. Initialize a model service veloxml init my-model cd my-model # 2. Deploy to AWS with scale-to-zero veloxml deploy # Output: # Replica ready at http://34.201.45.12:8000 # Test your endpoint: # curl -X POST http://34.201.45.12:8000/predict \ # -H "Content-Type: application/json" \ # -d '{"prompt": "Hello world"}'…from github.com
Does a similar job
all alternatives →- MLMy LLM CLI tool can run tools now, from Python code or plugins2025 · simonwillison.net · ▲529
- TCTiny CLI to save AWS costs in dev environments when you're sleeping2020 · npmjs.com · ▲119
- PDPipelex – Declarative language for repeatable AI workflowsOct 2025 · github.com · ▲122
We’re Robin, Louis, and Thomas. Pipelex is a DSL and a Python runtime for repeatable AI workflows. Think Dockerfile/SQL for multi-step LLM pipelines: you declare steps and interfaces; any model/provider can fill them. Why this instead of yet another workflow builder? - Declarative, not glue code: you state what to do; the runtime figures out how. - Agent-first: each step carries natural-language context (purpose, inputs/outputs with meaning) so LLMs can follow, audit, and optimize. Our MCP server enables agents to run pipelines but also to build new pipelines on demand. - Open…
- LTLLM-Term – Simple Rust-based CLI assist tool2024 · github.com · ▲32
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
- LALLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)2023 · github.com · ▲45
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
More dev tools this month
the category →
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 7d ago · opentrailpaper.com
Vendo ▲362Let your users build their own features inside your product
Dev tools · 25d ago · vendo.run
Launched alongside, September 2026
the whole month →- WI
Special relativity simulator in which light crawls along at a human speed, so Lorentz contraction, aberration and Doppler shift happen at walking pace.
Life & fun · 2d ago · rivendell.dmitrybrant.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 7d ago · opentrailpaper.com
9 major streaming services cost $95.91/mo in 2021-03 and $154.41/mo today — up 61%, or $702 a year. Every change sourced.
Life & fun · 1d ago · honestlyranked.com
- OD