Needle: We Distilled Gemini Tool Calling into a 26M Model
Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model. It runs at 6000 tok/s prefill and 1200 tok/s decode on consumer devices. We were always frustrated by the little effort made towards building agentic models that run on budget phones, so we conducted investigations that led to an observation: agentic experiences are built upon tool calling, and massive models are overkill for it. Tool calling is fundamentally retrieval-and-assembly (match query to tool name, extract argument values, emit JSON), not reasoning. Cross-attention…
Needle: We Distilled Gemini Tool Calling into a 26M Model launched on May 12, 2026 with 776 votes, #5 of 20,538 launches that month and more than 99% of that year's launches.
In plain words
Needle is a 26-million-parameter open-source model designed for function calling and tool use on consumer devices like phones and watches. It achieves fast performance at 6000 tokens per second for prefill and 1200 for decode by using only attention and gating layers without traditional MLPs, treating tool calling as a retrieval-and-assembly task rather than reasoning. The model is intended for developers building agent-based applications on resource-constrained hardware.
written from the facts on this page · September 2026
From the sources
14MB foundation model for tiny devices; phones, wearables, smart home, and robots. - cactus-compute/needle
Needle 2 is an open 45M-parameter model for tool calling, device use and structured extraction. The whole model is a single 14MB binary that runs a full session in about 28MB of RAM. It is built on our Simple Attention Network findings, compressed to CQ2-bit with Cactus Quants, and baked into its own engine. On the benchmarks below, Needle 2 trades wins with other small models like FunctionGemma 270M, LFM2.5 230M and Apple FM, at 5x to 70x smaller, and 2 bits against their f16. This repository is the Python package: inference, LoRA fine-tuning, and export. pip install cactus-needle , describe your tools, and call them from Python. The inference engine is fetched once from Hugging Face and…from github.com
In the maker’s words, at launch
Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model. It runs at 6000 tok/s prefill and 1200 tok/s decode on consumer devices. We were always frustrated by the little effort made towards building agentic models that run on budget phones, so we conducted…
Does a similar job
all alternatives →
Needle2: 14MB agentic LLM for phones, wearables, smart home and robotsAug 2026 · cactuscompute.com · ▲537Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
- IRI Remade the Fake Google Gemini Demo, Except Using GPT-4 and It's Real2023 · sagittarius.greg.technology · ▲434
Smart model routing directly in Claude, Codex and CursorJun 2026 · github.com · ▲216We built a model router that plugs into coding agents (e.g. Claude Code, Codex, Cursor, etc.) and intelligently sends requests to the best model to serve them. Here's a quick demo of running it locally: https://www.youtube.com/watch?v=isKhAyivtfM. At Weave, we write most of our code with AI, and it's been getting more expensive. This came to a head when Opus 4.7 was released and, thanks to its tokenizer changes, our costs shot up. We knew we didn't need Opus for everything but we didn't want to lose out on the intelligence for the cases where you really need it. So we decided…
Mysti – Claude, Codex, and Gemini debate your code, then synthesizeDec 2025 · github.com · ▲216Hey HN! I'm Baha, creator of Mysti. The problem: I pay for Claude Pro, ChatGPT Plus, and Gemini but only one could help at a time. On tricky architecture decisions, I wanted a second opinion. The solution: Mysti lets you pick any two AI agents (Claude Code, Codex, Gemini) to collaborate. They each analyze your request, debate approaches, then synthesize the best solution. Your prompt → Agent 1 analyzes → Agent 2 analyzes → Discussion → Synthesized solution Why this matters: each model has different training and blind spots. Two perspectives catch edge cases one would miss. It's like pair…
Axe – A 12MB binary that replaces your AI frameworkMar 2026 · github.com · ▲227I built Axe because I got tired of every AI tool trying to be a chatbot. Most frameworks want a long-lived session with a massive context window doing everything at once. That's expensive, slow, and fragile. Good software is small, focused, and composable... AI agents should be too. Axe treats LLM agents like Unix programs. Each agent is a TOML config with a focused job. Such as code reviewer, log analyzer, commit message writer. You can run them from the CLI, pipe data in, get results out. You can use pipes to chain them together. Or trigger from cron, git hooks, CI. What Axe is: - 12MB…
More life & fun this month
the category →
A free city-building game in your browser. You are the mayor. You know what the city needs. Now get it past the zoning board, the neighbors, the appeals court and the next election.
Life & fun · 21h ago · permit.petercai.com
- IV
Life & fun · 13d ago · tanzaniaeducationproject.org
Turn a phone, tablet or TV into a sign. Type a message and it fills the screen: welcome signs, timers, Wi-Fi passwords and more. No app, no account, free.
Life & fun · 14d ago · bigwords.page
Play a free virtual guitar online with your keyboard or by touch. Slow songs down and see every chord, string and strum. Acoustic, classic, electric and bass.
Life & fun · 8d ago · guitoro.com
Carrier settings from iPhone, Pixel and Galaxy firmware, decoded and compared: APNs, VoLTE, 5G and Wi-Fi Calling per carrier, and what each build changed.
Life & fun · 12d ago · carrierexplode.com
Life & fun · 22d ago · space.bl2.net
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, May 2026
the whole month →
Parallel agents, diff reviewer, and multi-model comparisons
Dev tools · May 2026 · kilo.ai
🌱 Private, quiet space for thinking. Simple app for .md files. - zakirullin/files.md
Dev tools · May 2026 · github.com
