Alternatives
Products that do what Lumabri – What if LLMs worked like Napster? does
A while ago I started working on Colibrì to see if it was possible to run huge LLMs on a normal computer. The project grew far beyond what I expected, thanks in large part to the HackerNews community. That led me to a new question: What if we stopped thinking about one computer? This is the idea behind Lumabri. Instead of requiring a single machine to store and run an entire huge model, Lumabri treats a network of normal computers as a shared pool of resources. One machine might provide disk space, another compute, another a different part of the model. If a required block or expert isn’t…
- 1GG
A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal computer like mine," and above all, "I wonder if it would work without going into OOM on a computer like mine." So I started working with the help of agents to test this possibility. I started converting the model to int4, understanding MTP usage, and if possible implementing DSA for long context.…
Jul 2026 · github.com
- 2
Run huge MoE models from a swarm of peers, with the colibri engine. Pure C. - JustVugg/lumabri
23d ago · github.com
- 3
- 4YA
Built this for my LLM workflows - needed searchable, persistent memory that wouldn't blow up storage costs. I also wanted to use it locally for my research. It's a content-addressed storage system with block-level deduplication (saves 30-40% on typical codebases). I have integrated the CLI tool into most of my workflows in Zed, Claude Code, and Cursor, and I provide the prompt I'm currently using in the repo. The project is in C++ and the build system is rough around the edges but is tested on macOS and Ubuntu 24.04.
2025 · github.com
- 5
- 6RM
Dec 2025 · github.com
- 7L0
Hey HN, Lume is an open-source CLI for running macOS and Linux VMs on Apple Silicon. Since launch (https://news.ycombinator.com/item?id=42908061), we've been using it to run AI agents in isolated macOS environments. We needed VMs that could set themselves up, so we built that. Here's what's new in 0.2: *Unattended Setup* – Go from IPSW to a fully configured VM without touching the keyboard. We built a VNC + OCR system that clicks through macOS Setup Assistant automatically. No more manual setup before pushing to a registry: lume create my-vm --os macos --ipsw latest…
Jan 2026 · cua.ai
- 8IB
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com
- 9KD
I've built this to make it easy to host your own infra for lightweight VMs at large scale. Intended for exec of AI-generated code, for CICD runners, or for off-chain AI DApps. Mainly to avoid Docker-in-Docker dangers and mess. Super easy to use with CLI / Python SDK, friendly to AI engs who usually don't like to mess with VM orchestration and networking too much. Defense-in-depth philosophy. Would love to get feedback (and contributors: clear & exciting roadmap!), thx
Oct 2025 · github.com
- 10
- 11LL
What it is A single 45 MB Windows .exe that embeds llama.cpp and a minimal Tk UI. Copy it (plus any .gguf model) to a flash drive, double-click on any Windows PC, and you’re chatting with an LLM—no admin rights, Cloud, or network. Why I built it Existing “local LLM” GUIs assume you can pip install, pass long CLI flags, or download GBs of extras. I wanted something my less-technical colleagues could run during a client visit by literally plugging in a USB drive. How it works PyInstaller one-file build → bundles Python runtime, llama_cpp_python, and the UI into a single PE. On first launch, it…
2025 · github.com
- 12
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right now I am running Qwen3-30B-A3B on my 24gb unified memory m4 MacBook Pro at 50 tok/sec and this should definitely not be working for such a large model on my middling hardware. Things are detailed in the README to get up and running and DESIGN.md has details on all the choices and such made along the way.
23d ago · github.com
- 13LI
Hey HN! We built Lunon to make LLM development way less of a headache. Ever wanted to see how different models handle the same prompt without all the setup hassle? That's what we fixed. Our API lets you compare Claude, GPT, Mistral and others in real-time with just a few lines of code. No more complex infrastructure or managing multiple API connections - we handle all that boring stuff behind the scenes. Plus, you can cut costs by intelligently routing requests to the right model for each task. Use the powerful (expensive) models only when you really need them. If you're building with LLMs…
2025 · lunon.com
- 14AM
I recently finished a round of tweaks on my barebones mud-server and thought HN might be interested. https://github.com/endgame/MudCore It sits somewhere between socketmud and a full mud like tbamud in that it handles all the socket work and telnet negotiation, but it makes no assumptions about anything else. The other interesting thing is that it can talk to other services using ZeroMQ. I'm interested in reactions, particularly in terms of coding style, documentation quality and so on. It's my main "presentation piece" so far, in that it's some of the first code I actually feel proud to…
2012
- 15TU
Over the past couple weeks, l've been really immersed in learning about MCP, a new protocol for equipping any LLM with a set of tools that can run on your own machine or a remote server you control and give all kinds of superpowers to Al agents to do things like search, etc. As part of that research, l've already built one very fleshed-out and useful MCP server that l've shared here (I've added much more to it recently though!), the LLM Gateway MCP Server, which lets you use a big model to delegate to a cheaper model (and many more things in addition to that, like running automated…
2025 · github.com
- 16R5
Hi HN, I built OpenGraviton, an open-source AI inference engine that pushes the limits of running extremely large LLMs on consumer hardware. By combining 1.58-bit ternary quantization, dynamic sparsity with Top-K pruning and MoE routing, and mmap-based layer streaming, OpenGraviton can run models far larger than your system RAM—even on a Mac Mini. Early benchmarks: TinyLlama-1.1B drops from ~2GB (FP16) to ~0.24GB with ternary quantization. At 140B scale, models that normally require ~280GB fit within ~35GB packed. Optimized for Apple Silicon with Metal + C++ tensor unpacking, plus…
Mar 2026 · github.com
- 17KP
I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edge devices like the Jetson Orin super nano kit which have unified memory. I haven't benched much, but plan to do so more over time and see if I can make a real use of it as I run local LLMs. Let me know if it makes sense :P (I of course vibed this idea)
Jun 2026 · github.com
- 18WB
Over the past few months, as we scaled our internal AI Agents, we hit a dead end: Running LLM-generated arbitrary code in Docker is basically running naked on security due to container escape risks. But using full traditional VMs takes minutes to boot and eats too much memory to support high-density concurrency. We loved the developer experience of SaaS sandboxes on the market, but they are closed-source, expensive, and have too high a barrier to entry for self-hosting. So, our team decided to build our own. After months of grinding, using RustVMM and KVM, we built a blazing-fast,…
Apr 2026 · github.com
- 19IB
hey hn, I built an open-source Perplexity clone that can run local LLMs and cloud LLMs. It's fully self-hostable through Docker and uses ollama to support local LLMs. The demo video in the repository shows me running it locally with llama3 on my M1 Macbook Pro. I'm open to any suggestions or feedback, thanks!
2024 · github.com
- 20PB
Started this a few months ago because I wanted a better test runner for an entirely different LLM-based project, then got completely nerdsniped by making LLMs as easy as possible to hotswap into other projects. It's still on the early side but it's finally at the point where I would happily use it for my original project, so I figured I'd post it. It's also—almost by accident—fully remote compatible (permissioning system included!), so you can host it on a box and then connect a program to it remotely.
2023 · github.com
- 21TF
i built TXTOS because my models kept forgetting and bluffing. i wanted a portable fix that works across providers without code or setup. TXTOS is a single .txt you paste into any LLM chat. it boots a small reasoning OS that gives you two things by default: a semantic tree memory that survives long threads, and a knowledge boundary guard that pushes back when the model is out of scope. what it is plain text. no scripts, no trackers, no api calls. MIT. the file encodes a protocol for reasoning, memory, and safety. you can diff it and fork it. it is not “a clever prompt”. it behaves like a tiny…
2025 · github.com
- 22CA
Hi HN, I've been working with LLMs in production for a while both as a solo dev building apps for clients and working at an AI startup. The one thing that always was a pain was to pay OpenAI/Gemini/Anthropic a few dollars a month just for me to say "test" or have a CI runner validate some UI code. So I built this server called ChunkBack, that mocks the popular llm provider's functionality but allows you to type in a deterministic language: `SAY "cheese"` or `TOOLCALL "tool_name" {} "tool response"` I've had to work in some test environments and give good results for experimenting…
Nov 2025 · github.com
- 23MA
Hi HN, A couple weeks ago I shared an early version of a side project I’ve been tinkering with called Persistent Mind Model. I built it at home on an i7-10700K / 32GB RAM / RTX 3080 because I was curious whether an AI could keep a stable “mind” over time, that could "think" about it's own identity as an LLM, instead of resetting every session. After a lot more tinkering, I think the architecture is finally in a solid place. Basically, it saves everything the AI does, thoughts, decisions, updates as a chain of events in a local SQLite database. Because the “identity” is stored in…
Nov 2025 · github.com
- 24XR
Hi HN, We built Xybrid, a Rust library for running LLM + speech pipelines directly inside your app, no server, no daemon, just one binary. We started building it while working on a privacy-focused LLM app with Tauri and realized there wasn’t a straightforward way to embed models directly into shipped applications without relying on a separate server process. Xybrid links into your process like any other library. It supports GGUF / ONNX / CoreML and integrates with Flutter, Swift, Kotlin, Unity, and Tauri, letting you run pipelines like speech → LLM → speech in a single call. On…
Mar 2026 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →