AI · alternatives · 2026
24 alternatives to Needle2: 14MB agentic LLM for phones, wearables, smart home and robots
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
Needle 2 is an open-source 45-parameter language model compressed to 14MB that performs tool calling, device control, and structured data extraction. Designed for resource-constrained devices like phones, smartwatches… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.
- 1
Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model. It runs at 6000 tok/s prefill and 1200 tok/s decode on consumer devices. We were always frustrated by the little effort made towards building agentic models that run on budget phones, so we conducted investigations that led to an observation: agentic experiences are built upon tool calling, and massive models are overkill for it. Tool calling is fundamentally retrieval-and-assembly (match query to tool name, extract argument values, emit JSON), not reasoning. Cross-attention…
May 2026 · github.com · its alternatives →
- 2
Needle 2.0▲582Vibe-automate workflows and earn passive income
Mar 2026 · needle.app · its alternatives →
- 3
Hey HN, Henry and Roman here - we've been building a cross-platform framework for deploying LLMs, VLMs, Embedding Models and TTS models locally on smartphones. Ollama enables deploying LLMs models locally on laptops and edge severs, Cactus enables deploying on phones. Deploying directly on phones facilitates building AI apps and agents capable of phone use without breaking privacy, supports real-time inference with no latency, we have seen personalised RAG pipelines for users and more. Apple and Google actively went into local AI models recently with the launch of Apple Foundation Frameworks…
2025 · github.com · its alternatives →
- 4
The stack: two agents on separate boxes. The public one (nullclaw) is a 678 KB Zig binary using ~1 MB RAM, connected to an Ergo IRC server. Visitors talk to it via a gamja web client embedded in my site. The private one (ironclaw) handles email and scheduling, reachable only over Tailscale via Google's A2A protocol. Tiered inference: Haiku 4.5 for conversation (sub-second, cheap), Sonnet 4.6 for tool use (only when needed). Hard cap at $2/day. A2A passthrough: the private-side agent borrows the gateway's own inference pipeline, so there's one API key and one billing relationship…
Mar 2026 · georgelarson.me · its alternatives →
- 5
Agentmemory▲310Persistent memory for Claude Code, Codex & coding agents
May 2026 · agent-memory.dev · its alternatives →
- 6
I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1]…
2024 · github.com · its alternatives →
- 7Agent 37▲401
Hey HN, I noticed there are 1,000+ MCP skills on GitHub with zero way for creators to charge for them. So I built Agent37. Selling Claude skills today means asking customers to download your skill, set up Claude Code on their laptops, configure MCP servers, and pray it works. No trial, no updates, often you're just handing over the source. Agent37 lets creators upload a skill, share a link, and anyone can try it instantly (no Claude account needed). Built-in Stripe, creators keep 80%. Looking for feedback on the approach. Would you use something like this?
Mar 2026 · agent37.com · its alternatives →
- 8
Hey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-trained to know when it's wrong. Every response comes with a confidence score between 0 and 1. Developers can accept the on-device when it's high, hand off to a bigger cloud model when it's low. By routing only 15-35% of queries to Gemini 3.1 Flash-Lite, Gemma-4-E2B matches Gemini 3.1 Flash-Lite on most benchmarks. - ChartQA: 15-20% - LibriSpeech: 25-30% - MMBench, GigaSpeech, MMAU: 30-35% -…
Jul 2026 · github.com · its alternatives →
- 9
We built a model router that plugs into coding agents (e.g. Claude Code, Codex, Cursor, etc.) and intelligently sends requests to the best model to serve them. Here's a quick demo of running it locally: https://www.youtube.com/watch?v=isKhAyivtfM. At Weave, we write most of our code with AI, and it's been getting more expensive. This came to a head when Opus 4.7 was released and, thanks to its tokenizer changes, our costs shot up. We knew we didn't need Opus for everything but we didn't want to lose out on the intelligence for the cases where you really need it. So we decided…
Jun 2026 · github.com · its alternatives →
- 10
Pi Coding Agent▲176A new Emacs mode for AI-assisted coding (Claude Code style), with native Emacs buffers instead of a flickering terminal. Built on top of pi, a no-bs open source Claude Code alternative. It supports Claude, GPT, Gemini, Ollama. I've been using it daily for weeks. Demo video that says more than $NUMBER words: https://danielnouri.org/media/pi-coding-agent-demo.mp4 The key decision here is that we have two separate windows: chat history is a markdown-mode buffer, with beautiful rendering, unlimted scrollback, selective tool expansion where it matters, etc. The input window…
May 2026 · pi.dev · its alternatives →
- 11
Context Gateway▲220We built an open-source proxy that sits between coding agents (Claude Code, OpenClaw, etc.) and the LLM, compressing tool outputs before they enter the context window. Demo: https://www.youtube.com/watch?v=-vFZ6MPrwjw#t=9s. Motivation: Agents are terrible at managing context. A single file read or grep can dump thousands of tokens into the window, most of it noise. This isn't just expensive — it actively degrades quality. Long-context benchmarks consistently show steep accuracy drops as context grows (OpenAI's GPT-5.4 eval goes from 97.2% at 32k to 36.6% at 1M…
Mar 2026 · github.com · its alternatives →
- 12
Hi HN! Erik here from Pig.dev, and today I'd like to share a new project we've just open sourced: Muscle Mem is an SDK that records your agent's tool-calling patterns as it solves tasks, and will deterministically replay those learned trajectories whenever the task is encountered again, falling back to agent mode if edge cases are detected. Like a JIT compiler, for behaviors. At Pig, we built computer-use agents for automating legacy Windows applications (healthcare, lending, manufacturing, etc). A recurring theme we ran into was that businesses already had RPA (pure-software scripts), and…
2025 · github.com · its alternatives →
- 13
Turing Pi is a compact ARM cluster that provides a secure and scalable compute in the edge. It is designed to make web-scale edge computing easier for developers.
2020 · turingpi.com · its alternatives →
- 14
The Emotion Engine has 32 MB of RAM total, so the trick is streaming weights from CD-ROM one matrix at a time during the forward pass — only activations, KV cache and embeddings live in RAM. This means models bigger than the RAM can still run, they just read more from disc. Had to build a custom quantized format (PSNT), hack endianness, write a tokenizer pipeline, and most of the PS2 SDK from scratch (releasing that separately). The model itself is also custom — a 10M param Llama-style architecture I trained specifically for this. And it works. On real hardware.
Mar 2026 · github.com · its alternatives →
- 15
PaperPodfree tier, from $0.0001/mo▲85Hi HN, I’ve been working on PaperPod, an experiment in making compute native to AI agents. PaperPod is a sandbox runtime where coding agents like Claude Code can autonomously execute code, write files, run background processes, expose ports (preview URLs) etc. and pay per ms of usage via the agent native x402 payment protocol. No API keys or complex setup needed. This is early (alpha), but fully usable today with free compute included, no wallets needed, just use your email as X-User-Id in the request! A sample prompt to get started: To use PaperPod compute: 1. Get session: POST…
Mar 2026 · paperpod.dev · its alternatives →
- 16AT
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260. Zero DRAM in the token loop, 59,965 tok/s on the fabric, bit-exact. Chat with it live.
Aug 2026 · mikeayles.com · its alternatives →
- 17
Hi all! This morning, we released a new Apache 2.0 licensed model on HuggingFace for detecting hallucinations in retrieval augmented generation (RAG) systems. What we've found is that even when given a "simple" instruction like "summarize the following news article," every LLM that's available hallucinates to some extent, making up details that never existed in the source article -- and some of them quite a bit. As a RAG provider and proponents of ethical AI, we want to see LLMs get better at this. We've published an open source model, a blog more thoroughly describing our methodology (and…
2023 · vectara.com · its alternatives →
- 18
Hi HN! My name is Olaf, I work at CircleCI as a technology advisor in the CTO office, came in through the acquisition of my company Vamp.io (progressive delivery for microservices on k8s) in 2021. Wanted to hear the HN community feedback and thoughts on a project we think could be very interesting when adding AI coding agents to the SDLC and your CI pipelines. Our team at CircleCI built Chunk sidecars after repeatedly running into the same issue internally: by the time our CI catches a failure, the agent has already moved on and most of the useful context is gone. The basic idea of Chunk…
May 2026 · circleci.com · its alternatives →
- 19
Hi HN, we're Kiran and Vijay! Over the past two years, we have built a columnar storage engine for observability: logs, metrics, and traces. Today, it's exciting for us to show what we've built on top of that foundation: LLM Agent Observability. Given how non-deterministic agents are, storing all traces without sampling was critical for us. But these traces tend to be in the MBs, sometimes GBs - we needed to store them inexpensively. We also needed the queries and analyses to be fast. To meet both these goals, we store them in S3 in our own parquet-like file format, and query them using AWS…
Jul 2026 · oodle.ai · its alternatives →
- 20AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
- 21W1
Hey HackerNews, I built this project over the last few weeks as a palette cleanser from a failed game launch. I wanted to learn a bit about AI/Neural-Networks and naively thought I could build a tiny maze-solving AI in a weekend with a 100% solve rate. Well - I couldn't, but I got pretty close. 14 Bytes total model size, and a 96.5% solve rate on unseen mazes. Trained across 46 phases experimenting with different ideas to improve the model (better performance, smaller size). Its quite fun to watch the model attempt to solve the maze, when they fail its usually due to getting stuck in a…
Jul 2026 · con-dog.github.io · its alternatives →
- 22
Needle▲12Enterprise-driven, adaptive, automated GenAI Framework
2024 · softwebsolutions.com · its alternatives →
- 23TF
I’d originally launched my app: Private LLM[1][2] on HN around 10 months ago, with a single RedPajama Chat 3B model. The app has come a long way since then. About a month ago, I added support for 4-bit OmniQuant quantized Mixtral 8x7B Instruct model, and it seems to outperform Q4 models at inference speed and Q8 models at text generation quality, while consuming only about 24GB of RAM[3] at 8k context length. The trick is: a) to use a better quantization algorithm and b) to use unquantized embeddings and the MoE gates (the overhead is quite small). Other notable features include many more…
2024 · its alternatives →
- 24BA
Hi HN, Erik here. Today we launch Butter, an OpenAI-compatible API proxy that caches LLM generations and serves them deterministically on revisit. Since April, we’ve been working on this concept of “muscle memory,” or deterministic replay, for agent systems performing automations. You may recall our first post in May, launching a python package called Muscle Mem: https://news.ycombinator.com/item?id=43988381 Since then, the product has evolved entirely, now taking the form of an LLM Proxy. For a deep dive into this process, check out:…
2025 · docs.butter.dev · its alternatives →
Also compare
- Needle: We Distilled Gemini Tool Calling into a 26M Model alternatives
- Needle 2.0 alternatives
- Cactus – Ollama for Smartphones alternatives
- I put an AI agent on a $7/month VPS with IRC as its transport layer alternatives
- Agentmemory alternatives
- Llama 3.2 Interpretability with Sparse Autoencoders alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →