AI · alternatives · 2026
24 alternatives to BonzAI – self-sovereign, local LLM inference in the browser
Hi everyone, I started building BonzAI two years ago, as an attempt to fully replace my online AI stack with an offline, sovereign alternative. Back in the days, I had the intuition that open weights model would stand the comparison with private AI labs, and that the commoditization of inference was the future. When I was finally able to switch from ChatGPT/Claude to local LLMs, midjourney to Flux/SDXL/Z-image, Eleven Labs/Suno to Qwen-TTS/Ace-Step, and Sora to LTX, I implemented: - The ability to create datasets and fine-tune LoRAs; - An experimental 3D objects…
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.
- 1
PrismML builds 1-bit and ternary Bonsai models that pack advanced reasoning, tool use, and multimodal AI into phones, laptops, and efficient data centers.
Mar 2026 · prismml.com · its alternatives →
- 2
I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1]…
2024 · github.com · its alternatives →
- 3
TensorZero▲307Hi HN! We're Gabriel & Viraj, and we're excited to open source TensorZero. To be a little cheeky, TensorZero is an open-source platform that helps LLM applications graduate from API wrappers into defensible AI products. 1. Integrate our model gateway 2. Send metrics or feedback 3. Unlock compounding improvements in quality, cost, and latency It enables a data & learning flywheel for LLMs by unifying: • Inference: one API for all LLMs, with <1ms P99 overhead • Observability: inference & feedback → your database • Optimization: better prompts, models, inference strategies • Experimentation:…
2025 · github.com · its alternatives →
- 4
LLM Gatewayfree tier▲296I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…
2025 · llmgateway.io · its alternatives →
- 5
Hey HN, I am proud to show you guys that I have built an open source alternative to Azure OpenAI services. Azure OpenAI services was born out of companies needing enhanced security and access control for using different GPT models. I want to build an OSS version of Azure OpenAI services that people could self host in their own infrastructure. "How can I track LLM spend per API key?" "Can I create a development OpenAI API key with limited access for Bob?" "Can I see my LLM spend breakdown by models and endpoints?" "Can I create 100 OpenAI API keys that my students could use in a classroom…
2023 · github.com · its alternatives →
- 6
SelfHostLLM▲134A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.
2025 · selfhostllm.org · its alternatives →
- 7
Hey HN, We've been building a multi-model API for LLMs called OpenRouter since we launched the Window AI extension in April this year [0]. The events of the last week make it clear the LLM landscape is unpredictable, so LLM aggregators are picking up interest. Unlike others, we're making a router on top of a public, explorable dataset. We recently built a way to rank and visualize this data, including the token counts we see going to and from different models, both open-sourced (like Llama, Mistral, finetunes, and variants) and closed-sourced (OpenAI, Anthropic). The API supports - 50+…
Apr 2026 · openrouter.ai · its alternatives →
- 8
Axis One▲78I’ve been building a small hardware box that runs local LLMs like Mistral, Qwen and Llama, and exposes an OpenAI compatible API on your local network. There’s no cloud, no login system and no telemetry. I built it because a lot of small firms want ChatGPTbstyle tools but can’t use cloud AI for privacy or compliance reasons, and most don’t want to deal with GPU servers, drivers, Docker or model configs. The aim is to make local AI feel as simple as plugging in a router. Right now the box boots into a very simple web UI where you choose a model and start using it. The API follows the OpenAI…
Nov 2025 · axis-one-psi.vercel.app · its alternatives →
- 9
We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
2025 · github.com · its alternatives →
- 10
Hi everyone, we’re a small team, supported by Mozilla, who are working on re-imagining a UI for training, tuning and testing local LLMs. Everything is open source. If you’ve been training your own LLMs or have always wanted to, we’d love for you to play with the tool and give feedback on what the future development experience for LLM engineering could look like.
2025 · github.com · its alternatives →
- 11
I keep running in the same problem of each AI app “remembers” me in its own silo. ChatGPT knows my project details, Cursor forgets them, Claude starts from zero… so I end up re-explaining myself dozens of times a day across these apps. The deeper problem 1. Not portable – context is vendor-locked; nothing travels across tools. 2. Not relational – most memory systems store only the latest fact (“sticky notes”) with no history or provenance. 3. Not yours – your AI memory is sensitive first-party data, yet you have no control over where it lives or how it’s queried. Demo video:…
2025 · github.com · its alternatives →
- 12
- 13
G'day, HN! I'm one of the maintainers of `llm`. I've been working alongside a trusty group of contributors to bring this project to life, and we're now at a point where we're ready to share it with the world. Large language models (LLMs) are taking the computing world by storm due to their emergent abilities that allow them to perform a wide variety of tasks, including translation, summarization, code generation, and even some degree of reasoning. However, the ecosystem around LLMs is still in its infancy, and it can be difficult to get started with these models. `llm` is a one-stop shop for…
2023 · github.com · its alternatives →
- 14RP
The excitement surrounding PrismML’s 1-bit/ternary Bonsai models has the industry closely watching how smartphone giants, particularly Apple, will implement LLMs on edge devices. Moving AI on-device is a brilliant and necessary strategy. It ensures absolute user privacy in alignment with EU regulations, fundamentally shifts the economics away from costly cloud inference, and paves the way for a significant hardware upgrade supercycle as users seek true AI-capable silicon. To create a smart on-device "Semantic Router," models need to reach the 27B+ parameter scale. Achieving this on a…
Jul 2026 · its alternatives →
- 15
I am working on a modular open source framework called Griptape that allows Python developers to create LLM pipelines and DAGs for complex workflows that use rules and memory. Griptape can be thought of as "Airflow for LLMs," providing an alternative to the agent-based LangChain approach. Developers can also build reusable LLM tools with explicit JSON schemas that can be executed in any environment (local, containerized, cloud, etc.) and integrated into Griptape workflows. They can also be easily converted into ChatGPT Plugin APIs and LangChain tools via adapters. Tools can be thought of as…
2023 · github.com · its alternatives →
- 16
Turn any document into an engaging video course in 15 min
2025 · bonzaiai.com · its alternatives →
- 17
Hey HN, I've been building AutoAgents, an AI agent framework in Rust. Today I'm sharing a feature I haven't seen done well elsewhere: composable middleware layers for LLM inference pipelines. The problem Every agent framework lets you swap LLM providers. Almost none of them give you a structured way to enforce safety, caching, or data sanitization in the inference path itself. You end up with guardrails as application-level if-statements, caching bolted on as a separate service, and PII handling as a "we'll add it later" TODO that never ships. This gets worse with local models. Cloud APIs…
Mar 2026 · github.com · its alternatives →
- 18
- 19
An educational Rust project for exporting and running inference on Qwen3 LLM family - reinterpretcat/qwen3-rs
2025 · github.com · its alternatives →
- 20
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
2024 · featherless.ai · its alternatives →
- 21
Introducing Bonsai 0.5B, one of the first ternary-weight LLMs to rival full-precision models of similar size, such as Qwen 2.5 0.5B and MobileLLM 0.5B. Trained on just 3.8B tokens, using 1,000x less data than other models, Bonsai redefines what’s possible for ultra-efficient training in low-bit models. Next, we're building larger and more powerful ternary-weight models for the edge. Technical Report: https://github.com/deepgrove-ai/Bonsai/blob/main/paper/Bonsa... Model (Unpacked): https://huggingface.co/deepgrove/Bonsai Reach us:…
2025 · github.com · its alternatives →
- 22IM
I’m Hayden, a 13-year-old developer based in Australia, and I’ve built a CoT logical thinking and reasoning AI model similar to OpenAI o1. It's powered by open source small models like Llama 3.1 and 3.2 and I would love for you to try it and share your feedback with me. You can try it here: https://ai.pixelverse.tech/app/cortexchat I built it just for fun and launched it a day after the o1 release. It's not perfect yet but its still amazing to see how a detailed prompt can have such a difference on the quality of the LLM response! Please let me know any feedback or…
2024 · ai.pixelverse.tech · its alternatives →
- 23
I’ve been building LLM tooling for a small VC fund and found myself explaining the same mental model over and over to non-technical people around me: how a stateless LLM becomes a chatbot, how tool use works, what an agent is mechanically, and why context windows shape all of it. I never found a guide that covered that full chain at the level I wanted, so I wrote one. It’s nine short chapters, each building on the last. Deliberately simplified: the goal is a useful mental model, not a textbook. Feedback, corrections, and contributions welcome: github.com/ymyke/aiaiai
Apr 2026 · aiaiai.guide · its alternatives →
- 24
I've built an airgapped Retrieval-Augmented Generation (RAG) system for question-answering on documents, running entirely offline with local inference. Using Llama 3, Mistral, and Gemini, this setup allows secure, private NLP on your own machine. Perfect for researchers, data scientists, and developers who need to process sensitive data without cloud dependencies. Built with Llama C++, LangChain, and Streamlit, it supports quantized models and provides a sleek UI for document processing. Check it out, contribute, or suggest new features!
2024 · github.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →