Bonsai 1.7B ternary model at 442T/s on M4 Max
We took a recently released Bonsai 1.7B ternary model from PrismML (https://github.com/PrismML-Eng/Bonsai-demo) and ran our agentic evolution search on it for 6 hours to optimize the Metal kernels. The search was fully autonomous. Measured against unmodified upstream llama.cpp at the same Bonsai/Q2_0 commit, same M4 Max: - tg128: 309.82 → 442.42 t/s (+42.0%) - pp512: 4250.32 → 4622.63 t/s (+8.8%)
Bonsai 1.7B ternary model at 442T/s on M4 Max launched on May 4, 2026 with 13 votes, #1,061 of 20,538 launches that month and more than 90% of that year's launches.
In plain words
This is an optimized implementation of the Bonsai 1.7B ternary language model running on Apple's M4 Max chip. Using automated agentic evolution search, the developers optimized Metal GPU kernels to achieve 42% faster token generation and 8.8% faster prompt processing compared to the unmodified upstream version. It is designed for developers working with efficient language models on Mac hardware.
written from the facts on this page · September 2026
From the sources
A drop-in optimized inference build of Bonsai 1.7B for Apple Silicon. ~42% faster decode (tg128) and ~9% faster prefill (pp512) on M-series GPUs. All Metal kernels written and tuned by ata, our autonomous engineering agent.
Does a similar job
all alternatives →
1-Bit Bonsai, the First Commercially Viable 1-Bit LLMsMar 2026 · prismml.com · ▲430PrismML builds 1-bit and ternary Bonsai models that pack advanced reasoning, tool use, and multimodal AI into phones, laptops, and efficient data centers.
Bonsai – A Competitive Ternary Weight LLM2025 · github.com · ▲11Introducing Bonsai 0.5B, one of the first ternary-weight LLMs to rival full-precision models of similar size, such as Qwen 2.5 0.5B and MobileLLM 0.5B. Trained on just 3.8B tokens, using 1,000x less data than other models, Bonsai redefines what’s possible for ultra-efficient training in low-bit models. Next, we're building larger and more powerful ternary-weight models for the edge. Technical Report: https://github.com/deepgrove-ai/Bonsai/blob/main/paper/Bonsa... Model (Unpacked): https://huggingface.co/deepgrove/Bonsai Reach us:…
- RPRunning PrismML's Bonsai inside DRAM by breaking DDR4 timing rulesJul 2026 · ▲23
The excitement surrounding PrismML’s 1-bit/ternary Bonsai models has the industry closely watching how smartphone giants, particularly Apple, will implement LLMs on edge devices. Moving AI on-device is a brilliant and necessary strategy. It ensures absolute user privacy in alignment with EU regulations, fundamentally shifts the economics away from costly cloud inference, and paves the way for a significant hardware upgrade supercycle as users seek true AI-capable silicon. To create a smart on-device "Semantic Router," models need to reach the 27B+ parameter scale. Achieving this on a…
- WBWeb browser to help programmers think clearly2021 · bonsaibrowser.com · ▲1,070
- MPMaple-Preview – Ternary 20B MoE running at 120 tok/s on a iPhoneAug 2026 · deepgrove.ai · ▲173
80% faster, 50% less memory, 0% loss of accuracy Llama finetuning2023 · github.com · ▲385Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
More dev tools this month
the category →
Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.
Dev tools · 2d ago · github.com
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 30d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 28d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 25d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 23d ago · talanoa.email
Source: Show HN post ↗
Launched alongside, May 2026
the whole month →
Parallel agents, diff reviewer, and multi-model comparisons
Dev tools · May 2026 · kilo.ai
Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model. It runs at 6000 tok/s prefill and 1200 tok/s decode on consumer devices. We were always frustrated by the little effort made towards building agentic models that run on budget phones, so we conducted investigations that led to an observation: agentic experiences are built upon tool calling, and massive models are overkill for it. Tool calling is fundamentally retrieval-and-assembly (match query to tool name, extract argument values, emit JSON), not reasoning. Cross-attention…
Life & fun · May 2026 · github.com
🌱 Private, quiet space for thinking. Simple app for .md files. - zakirullin/files.md
Dev tools · May 2026 · github.com