AI · alternatives · 2026

PrismML's Bonsai is a technique for running large language models directly on smartphones by using extreme quantization and optimizing memory access patterns in DRAM. It enables 27+ billion parameter models on mobile… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.

1. 1
   1-Bit Bonsai, the First Commercially Viable 1-Bit LLMs▲430

   PrismML builds 1-bit and ternary Bonsai models that pack advanced reasoning, tool use, and multimodal AI into phones, laptops, and efficient data centers.

   Mar 2026 · prismml.com · its alternatives →

2. 2
   Bonsai 1.7B ternary model at 442T/s on M4 Max▲13

   We took a recently released Bonsai 1.7B ternary model from PrismML (https://github.com/PrismML-Eng/Bonsai-demo) and ran our agentic evolution search on it for 6 hours to optimize the Metal kernels. The search was fully autonomous. Measured against unmodified upstream llama.cpp at the same Bonsai/Q2_0 commit, same M4 Max: - tg128: 309.82 → 442.42 t/s (+42.0%) - pp512: 4250.32 → 4622.63 t/s (+8.8%)

   May 2026 · agents2agents.ai · its alternatives →

3. 3
   Cactus – Ollama for Smartphones▲231

   Hey HN, Henry and Roman here - we've been building a cross-platform framework for deploying LLMs, VLMs, Embedding Models and TTS models locally on smartphones. Ollama enables deploying LLMs models locally on laptops and edge severs, Cactus enables deploying on phones. Deploying directly on phones facilitates building AI apps and agents capable of phone use without breaking privacy, supports real-time inference with no latency, we have seen personalised RAG pipelines for users and more. Apple and Google actively went into local AI models recently with the launch of Apple Foundation Frameworks…

   2025 · github.com · its alternatives →

4. 4
   Llama 3.2 Interpretability with Sparse Autoencoders▲579

   I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1]…

   2024 · github.com · its alternatives →

5. 5
   Ollama v0.19▲410

   Massive local model speedup on Apple Silicon with MLX

   Apr 2026 · ollama.com · its alternatives →

6. 6
   Bonsai▲105

   AI programming platform for enterprises

   2017 · its alternatives →

7. 7
   Needle2: 14MB agentic LLM for phones, wearables, smart home and robots▲537

   Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…

   Aug 2026 · cactuscompute.com · its alternatives →

8. 8
   RunAnywhere▲119

   Ollama but for mobile, with a cloud fallback

   2025 · runanywhere.ai · its alternatives →

9. 9
   Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone▲312

   Swiftlet is a Swift and Metal runtime that runs large Qwen Mixture-of-Experts models locally on Apple devices by streaming expert weights from storage, enabling 35B and 80B models to run with low RAM, including on iPhone. - leonickson1/Swiftlet

   Aug 2026 · github.com · its alternatives →

10. 10
    80% faster, 50% less memory, 0% loss of accuracy Llama finetuning▲385

    Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…

    2023 · github.com · its alternatives →

11. 11
    LLM Gatewayfree tier▲296

    I built an OpenAI-compatible LLM gateway that routes requests to OpenAI, Anthropic, Ollama, vLLM, llama-server, SGLang... anything that speaks /v1/chat/completions. Single Go binary, one YAML config file, no infrastructure. It does the things you'd expect from this kind of gateway... semantic routing via a three-layer cascade (keyword heuristics, embedding similarity, LLM classifier) that picks the best model when clients omit the model field, weighted round-robin load balancing across local inference servers with health checks and failover. The part I think is most…

    2025 · llmgateway.io · its alternatives →

12. 12
    Bonsai – A Competitive Ternary Weight LLM▲11

    Introducing Bonsai 0.5B, one of the first ternary-weight LLMs to rival full-precision models of similar size, such as Qwen 2.5 0.5B and MobileLLM 0.5B. Trained on just 3.8B tokens, using 1,000x less data than other models, Bonsai redefines what’s possible for ultra-efficient training in low-bit models. Next, we're building larger and more powerful ternary-weight models for the edge. Technical Report: https://github.com/deepgrove-ai/Bonsai/blob/main/paper/Bonsa... Model (Unpacked): https://huggingface.co/deepgrove/Bonsai Reach us:…

    2025 · github.com · its alternatives →

13. 13
    MiniCPM 4.1▲116

    The on-device model for your personal data

    2025 · huggingface.co · its alternatives →

14. 14
    We made our own inference engine for Apple Silicon▲186

    We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.

    2025 · github.com · its alternatives →

15. 15AT
    A tiny LLM running at 21,000 tok/s on a $250 FPGA (Live Demo)▲79

    A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260. Zero DRAM in the token loop, 59,965 tok/s on the fabric, bit-exact. Chat with it live.

    Aug 2026 · mikeayles.com · its alternatives →

16. 16
    Tiny-vLLM – high performance LLM inference engine in C++ and CUDA▲205

    Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm

    May 2026 · github.com · its alternatives →

17. 17
    SelfHostLLM▲134

    A simple calculator that estimates how many concurrent requests your GPU can handle for a given LLM, with shareable results.

    2025 · selfhostllm.org · its alternatives →

18. 18
    We built a knowledge hub for running LLMs on edge devices▲13

    Hey HN! Alex and Zack from Nexa AI here. We are excited to share a project our team has been passionately working on recently, in collaboration with Jiajun from Meta, Qun from San Francisco State University, and Xin and Qi from the University of North Texas. Running AI models on edge devices is becoming increasingly important. It's cost-effective, ensures privacy, offers low-latency responses, and allows for customization. Plus, it's always available, even offline. What's really exciting is that smaller-scale models are now approaching the performance of large-scale closed-source models for…

    2024 · github.com · its alternatives →

19. 19MP
    Maple-Preview – Ternary 20B MoE running at 120 tok/s on a iPhone▲173

    Aug 2026 · deepgrove.ai · its alternatives →

20. 20
    Fine-tune an 8B model on a 4 GB laptop GPU▲139

    Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU. - MakazhanAlpamys/Soup

    Aug 2026 · github.com · its alternatives →

21. 21
    MiniCPM 4.0▲110

    Ultra-efficient on-device AI, now even faster

    2025 · github.com · its alternatives →

22. 22
    Bonsai Image▲5

    Ultra-Fast, Light-as-Air AI Generation

    Jun 2026 · bonsaiimage.com · its alternatives →

23. 23
    Axis One▲78

    I’ve been building a small hardware box that runs local LLMs like Mistral, Qwen and Llama, and exposes an OpenAI compatible API on your local network. There’s no cloud, no login system and no telemetry. I built it because a lot of small firms want ChatGPTbstyle tools but can’t use cloud AI for privacy or compliance reasons, and most don’t want to deal with GPU servers, drivers, Docker or model configs. The aim is to make local AI feel as simple as plugging in a router. Right now the box boots into a very simple web UI where you choose a model and start using it. The API follows the OpenAI…

    Nov 2025 · axis-one-psi.vercel.app · its alternatives →

24. 24
    Core – open source memory graph for LLMs – shareable, user owned▲112

    I keep running in the same problem of each AI app “remembers” me in its own silo. ChatGPT knows my project details, Cursor forgets them, Claude starts from zero… so I end up re-explaining myself dozens of times a day across these apps. The deeper problem 1. Not portable – context is vendor-locked; nothing travels across tools. 2. Not relational – most memory systems store only the latest fact (“sticky notes”) with no history or provenance. 3. Not yours – your AI memory is sensitive first-party data, yet you have no control over where it lives or how it’s queried. Demo video:…

    2025 · github.com · its alternatives →

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →