Self-host open-source LLMs on AWS with scale-to-zero
Deploy LLMs to production in one command. VeloxML CLI is an open-source engine that provisions optimized serverless infrastructure directly in your own AWS/GCP account - paguasmar/veloxml-deploy
Self-host open-source LLMs on AWS with scale-to-zero launched on September 9, 2026 with 7 votes, #1,176 of 8,094 launches that month and more than 75% of that year's launches.
What it does
Deploy open-source LLMs directly to your own AWS/GCP account with one command. Zero Docker, zero Kubernetes, scale-to-zero. This repo is still under heavy development and the documentation is evolving. You're welcome to try it, but expect some breaking changes. Watch "releases" of this repo to receive a notification when we are ready for Beta. And give us a star if you like it! # 1. Initialize a model service veloxml init my-model cd my-model # 2. Deploy to AWS with scale-to-zero veloxml deploy # Output: # Replica ready at http://34.201.45.12:8000 # Test your endpoint: # curl -X POST http://34.201.45.12:8000/predict \ # -H "Content-Type: application/json" \ # -d '{"prompt": "Hello world"}'…from github.com
Does a similar job
all alternatives →
A CLI tool I made to self-host any app with two commands on a VPS2024 · github.com · ▲452Bare metal to production ready in mins; your own fly server on your VPS. - MightyMoud/sidekick
- TCTiny CLI to save AWS costs in dev environments when you're sleeping2020 · npmjs.com · ▲119
Pipelex – Declarative language for repeatable AI workflowsOct 2025 · github.com · ▲122We’re Robin, Louis, and Thomas. Pipelex is a DSL and a Python runtime for repeatable AI workflows. Think Dockerfile/SQL for multi-step LLM pipelines: you declare steps and interfaces; any model/provider can fill them. Why this instead of yet another workflow builder? - Declarative, not glue code: you state what to do; the runtime figures out how. - Agent-first: each step carries natural-language context (purpose, inputs/outputs with meaning) so LLMs can follow, audit, and optimize. Our MCP server enables agents to run pipelines but also to build new pipelines on demand. - Open…
LLM-Term – Simple Rust-based CLI assist tool2024 · github.com · ▲32Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
LLM, a Rust Crate/CLI for CPU Inference of LLMs (LLaMA, GPT-NeoX, etc.)2023 · github.com · ▲45G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
Llm2sh – Translate plain-language requests into shell commands2024 · github.com · ▲67This is my take on the common "use llms to generate shell commands" utility. Emphasis is placed on good CLI UX, simplicity, and flexibility. `llm2sh` supports multiple LLM providers and lets LLMs generate multi-command sequences to handle complex tasks. There is also limited support for commands requiring `sudo` and other basic input. I recommend using Groq llama3-70b for day-to-day use. The ultra-low latency is a game-changer - its near-instant responses helps `llm2sh` integrate seamlessly into day-to-day tasks without breaking you out of the 'zone'. For more advanced tasks, swapping to…
More dev tools this month
the category →
Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.
Dev tools · 5d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 28d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 26d ago · talanoa.email
Launched alongside, September 2026
the whole month →
E-ink bird frame for Raspberry Pi - real-time bird detection by audio, fully local AI, rendered as real, hand-cut 1800s bird illustrations. - arnegiacomo/fugleramme
AI · Sep 2026 · github.com
- WI
Special relativity simulator in which light crawls along at a human speed, so Lorentz contraction, aberration and Doppler shift happen at walking pace.
Life & fun · Sep 2026 · rivendell.dmitrybrant.com
Start coding agents, control sessions, review PRs. Anywhere.
Dev tools · Sep 2026 · kilo.ai
