Alternatives

Products that do what Needle: We Distilled Gemini Tool Calling into a 26M Model does

Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model. It runs at 6000 tok/s prefill and 1200 tok/s decode on consumer devices. We were always frustrated by the little effort made towards building agentic models that run on budget phones, so we conducted investigations that led to an observation: agentic experiences are built upon tool calling, and massive models are overkill for it. Tool calling is fundamentally retrieval-and-assembly (match query to tool name, extract argument values, emit JSON), not reasoning. Cross-attention…

1. Not indexed yet — check back in a few minutes.

Ranked by how close each launch is in meaning, then by votes. Refine with a description →