Alternatives
Products that do what BitTorrent-style LLMs come to Kalavai does
Today we are thrilled to announce the release of a public Kalavai pool dedicated to host Petals workers. This is part of our wider effort to offer easy access to compute to AI developers. Do you find this useful? What other tooling would you like to see running on crowdsourced hardware?
- 1
- 2IV
The video demo runs a 7b Model on a normal gaming GPU. I think it already works quite well (accounting for the limited hardware power). :)
2024 · github.com
- 3AT
I have a favour to ask. I’ve been working for a while on Kalavai, a project to make distributed AI easy. There are brilliant tools out there to help AI hobbyists and devs on the software layer (shout out to vLLM and llamacpp amongst many others!) but it’s a jungle out there when it comes to procuring and managing the necessary hardware resources and orchestrating them. This has always led me to compromise on the size of the models I end up using (quantized versions, smaller models) to save cost or to play within the limits of my rig. Today I am happy to share the first public version of our…
2024 · github.com
- 4
- 5IS
Everything that would be here is in the README. I hope this gets big, it has tons of potential.
2013 · github.com
- 6WB
Over the past few months, as we scaled our internal AI Agents, we hit a dead end: Running LLM-generated arbitrary code in Docker is basically running naked on security due to container escape risks. But using full traditional VMs takes minutes to boot and eats too much memory to support high-density concurrency. We loved the developer experience of SaaS sandboxes on the market, but they are closed-source, expensive, and have too high a barrier to entry for self-hosting. So, our team decided to build our own. After months of grinding, using RustVMM and KVM, we built a blazing-fast,…
Apr 2026 · github.com
- 7RA
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
2024 · featherless.ai
- 8IM
I spent the past few weeks making an open source cloud code editing environment with an AI copilot and multiplayer collaboration! It's fully self-hostable in 5-10 minutes. There's a lot of minor improvements to be made, and some are already listed in the Github issues. Let me know what you think and feel free to try it out.
2024 · github.com
- 9CA
Hi HN, I've been working with LLMs in production for a while both as a solo dev building apps for clients and working at an AI startup. The one thing that always was a pain was to pay OpenAI/Gemini/Anthropic a few dollars a month just for me to say "test" or have a CI runner validate some UI code. So I built this server called ChunkBack, that mocks the popular llm provider's functionality but allows you to type in a deterministic language: `SAY "cheese"` or `TOOLCALL "tool_name" {} "tool response"` I've had to work in some test environments and give good results for experimenting…
Nov 2025 · github.com
- 10SH
Hello everyone, I am the creator of Stable Horde, a free, open sourced, crowdsourced, distributed cluster of Stable Diffusion (AKA AI image generation) workers. This service is provided for free and without any small print. The only thing one needs to be aware of is that your prompt goes to someone else's computer to be created and then sent back with all the caveats that entails. The main difference from all the other Stable Diffusion webuis and services are: * This service is free for everyone, without credits. While we do have a kudos system, this is only used for priority and you can…
2022 · stablehorde.net
- 11LV
Hello HN! We've been working on a new hypervisor https://kwarantine.xyz that can run strongly isolated containers. This is still a WIP, but we wanted to give the community an idea about our approach, its benefits, and various use cases it unlocks. Today, VMs are used to host containers, and make up for the lack of strong security as well as kernel isolation in containers. This work adds this missing security piece in containers. We plan on launching a free private beta soon. Meanwhile, we'd deeply appreciate any feedback, and happy to answer any questions here or on our slack…
2021
- 12HA
2015 · github.com
- 13FC
Hi there, I've created this side project to make it easier to find interesting repositories using AI. There's still a lot of work to be done to improve it, so any suggestions for enhancements would be greatly appreciated. Thank you!
2024 · awesome-repositories.com
- 14EA
2023 · hyperaide.com
- 15IM
Hey HN! Thank you for all the support and feedback on my original submission 2 months ago. I've been improving the backend using a MCTS/AlphaZero approach and it's currently producing much better results. My long term goal is to allow users to manage multiple projects, deployed autonomously, both from scratch and by making continual updates all prompted with natural language. The cost of each project has been lowered to $9 as performance with smaller models has improved (I migrated from Claude-3-Opus to gemini-1.5-flash). Thanks for checking it out!
2024 · saas-quick.com
- 16IB
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com
- 17
- 18PA
Been working on data sovereignty recently and started this list. Hope you can contribute too.
2025 · github.com
- 19IB
Hi HN, I’m the creator of Cordum. I’ve been working in DevOps and infrastructure for years (currently in the fintech/security space), and as I started playing with AI agents, I noticed a scary pattern. Most "safety" mechanisms rely on system prompts ("Please don't do X") or flimsy Python logic inside the agent itself. If we treat agents as autonomous employees, giving them root access and hoping they listen to instructions felt insane to me. I wanted a way to enforce hard constraints that the LLM cannot override, no matter how "jailbroken" it gets. So I built Cordum. It’s an open-source…
Jan 2026 · github.com
- 20S1
I wanted to build an inference provider for proprietary AI models, but I did not have a huge GPU farm. I started experimenting with Serverless AI inference, but found out that coldstarts were huge. I went deep into the research and put together an engine that loads large models from SSD to VRAM up to ten times faster than alternatives. It works with vLLM, and transformers, and more coming soon. With this project you can hot-swap entire large models (32B) on demand. Its great for: Serverless AI Inference Robotics On Prem deployments Local Agents And Its open source. Let me know if anyone…
Nov 2025 · github.com
- 21IB
I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things. The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture,…
Apr 2026 · qwelian.com
- 22RS
Hi HN! I've been hacking on this side project for the last month or two with the goal of making it dead simple to use cloud GPUs. I ran into this problem personally during the phd, and built my own tooling around it. I always thought it'd be fun to try to turn that tooling into a more general product... and bitbop.io is the result! All you have to do is run `ssh bitbop.io`, and you get your own personal dev GPU workstation in the cloud. Looking forward to hearing your thoughts!
2024 · twitter.com
- 23AL
Hello all, a couple of moons ago I ported karpathy's llama2.c code to run inference on the TinyStories 260K & 15M checkpoints on the on the PS Vita with the ability to download/delete the models on device. Runs showed that the 260K model ran at ~120 tok/s and at 15M ran at 1.8 tok/s, which probably could be a bit higher if it weren't a single threaded application. Had fun working on it as a weekend project, check it out for yourselves: https://github.com/callbacked/psvita-llm
2025 · github.com
- 24IB
Hey HN! Over the past few weeks, I’ve been working on DataBridge, an open-source solution for data ingestion and querying across text, PDFs, images, and videos. In our latest update, we’ve added a fully local deployment option: - No internet required – Runs entirely offline. - Customizable Models – Supports any LLM and embedding model via Ollama (with options for any other private providers) - Extensibility – You can plug in your own models or tools easily. This local-first approach ensures better privacy, security, and flexibility, especially for teams dealing with sensitive data. You can…
2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →