Cactus Hybrid: We taught Gemma 4 to know when it's wrong

In plain words

Cactus Hybrid is a hybrid AI system that pairs a small on-device model with larger cloud models to balance speed, privacy, and accuracy. The on-device model, based on post-trained Gemma 4, provides a confidence score with each response, allowing developers to use local results when confidence is high and route uncertain queries to more capable models like Gemini 3.1 Flash-Lite. This approach reduces cloud model calls to 15-35% while matching larger model performance on benchmark tasks, offering a cost-effective solution for developers building AI applications.

written from the facts on this page · September 2026

From the sources

On-device models that know when they're wrong: every answer carries a confidence score for cloud handoff. - cactus-compute/cactus-hybrid

A small, on-device model is fast and private, but sometimes wrong. At Cactus we post-train models to know when they are wrong : we ship probes inside the checkpoint that score every answer with a confidence between 0 and 1, returned as structured data (never parsed out of the answer text). Answer on-device when confidence is high; you can re-route to a bigger model when it's low: if confidence < 0.85 : answer = ask_a_bigger_model ( prompt ) We start the rollout with Gemma 4 E2B Hybrid , all builds live in the Cactus Hybrid collection on Hugging Face. Gemma 4 E2B hybrid, the smallest Gemma model, matches Gemini 3.1 Flash-Lite on common benchmarks by routing only 15–55% of queries to the…from github.com

In the maker’s words, at launch

Hey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-trained to know when it's wrong. Every response comes with a confidence score between 0 and 1. Developers can accept…

• 
  Gemma 22024 · blog.google · ▲279

  Lightweight, state-of-the-art open models from Google

• 
  Google Gemma 4Apr 2026 · blog.google · ▲437

  Google's most intelligent open models to date

• 
  Cactus – Ollama for Smartphones2025 · github.com · ▲231

  Hey HN, Henry and Roman here - we've been building a cross-platform framework for deploying LLMs, VLMs, Embedding Models and TTS models locally on smartphones. Ollama enables deploying LLMs models locally on laptops and edge severs, Cactus enables deploying on phones. Deploying directly on phones facilitates building AI apps and agents capable of phone use without breaking privacy, supports real-time inference with no latency, we have seen personalised RAG pipelines for users and more. Apple and Google actively went into local AI models recently with the launch of Apple Foundation Frameworks…

• 
  Gemma2024 · deepmind.google · ▲259

  Google’s new state-of-the-art open source LLMs

• 
  Needle2: 14MB agentic LLM for phones, wearables, smart home and robotsAug 2026 · cactuscompute.com · ▲537

  Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens&#x2F;sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens&#x2F;sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…

• 
  Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series MacJul 2026 · github.com · ▲919

  Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal. I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory. The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are…

• 
  SimCity, but with Permits▲936

  A free city-building game in your browser. You are the mayor. You know what the city needs. Now get it past the zoning board, the neighbors, the appeals court and the next election.

  Life & fun · 3d ago · permit.petercai.com

• IV
  I've been paying for a rural Tanzanian's education for 10 years▲778

  Life & fun · 15d ago · tanzaniaeducationproject.org

• 
  Bigwords.page – Turn any screen into a sign. The URL is the app▲728

  Turn a phone, tablet or TV into a sign. Type a message and it fills the screen: welcome signs, timers, Wi-Fi passwords and more. No app, no account, free.

  Life & fun · 16d ago · bigwords.page

• 
  Guitoro, a virtual guitar you strum with your keyboard▲573

  Play a free virtual guitar online with your keyboard or by touch. Slow songs down and see every chord, string and strum. Acoustic, classic, electric and bass.

  Life & fun · 10d ago · guitoro.com

• 
  See *almost* any carriers phone settings▲432

  Carrier settings from iPhone, Pixel and Galaxy firmware, decoded and compared: APNs, VoLTE, 5G and Wi-Fi Calling per carrier, and what each build changed.

  Life & fun · 14d ago · carrierexplode.com

• 
  The whole Solar System and every tracked satellite, live in the browser▲407

  Life & fun · 24d ago · space.bl2.net

• IR
  I replaced a $120k bowling center system with $1,600 in ESP32s▲2,935

  I might be the only SRE on Earth with his own bowling center. It's a more in-depth gig than you'd think. My family and I bought an abandoned 8-lane bowling center in the rural mid-west. In our small town there weren't many recreation options for families. You've heard of a food desert? This is an R&R desert. It had been abandoned for a good reason. The roof leaks, the electrical system was constantly surging, and my 70-year-old bowling equipment (still) doesn't work perfectly. The system that keeps your score is particularly interesting to me. It's the thing you watch during your game, but…

  Life & fun · Jul 2026

• 
  Elevators▲1,680

  How elevators push your buttons

  Life & fun · Jul 2026 · john.fun

• 
  18 Words▲1,160

  Play 18 Words, the free daily word puzzle. The official home is 18words.com. Tap Play now to open today

  Life & fun · Jul 2026 · 18words.com

• BA
  Bento - An entire PowerPoint in one HTML file (edit+view+data+collab)▲1,033

  Over the past few months, our team has been building more and more slidedecks using web frontend technologies with coding harnesses like Claude Code, but a common complaint is to make even small edits we need to edit the code either manually or via the harness. To avoid this loop, I ended up creating Bento, a single HTML file with everything you need in a slide tool including animations and shared editing. There's no install or cloud login, everything works offline. The default deck is around 560 KB and it doesn't need to fetch anything once you got it. Open it in a browser and then you can…

  Dev tools · Jul 2026 · bento.page

• 
  Getting GLM 5.2 running on my slow computer▲937

  A few days ago I found myself trying out GLM 5.2 and was really positively impressed. The capabilities and security I was getting from this LLM are similar to those I've gotten from models like Claude or GPT, and this really surprised me. But then I thought, "I wonder how it would work on a normal computer like mine," and above all, "I wonder if it would work without going into OOM on a computer like mine." So I started working with the help of agents to test this possibility. I started converting the model to int4, understanding MTP usage, and if possible implementing DSA for long context.…

  AI · Jul 2026 · github.com

• 
  OpenSEO▲928

  Hello! For the last 2 months, I've been building OpenSEO. It's a user friendly UI wrapping the DataForSEO API which has high quality SEO data for keywords, backlinks, AI SEO and competitor research. Instead of paying $100+ per month, you buy DataForSEO credits and pay only for usage. The coolest thing about the project so far has been hearing about what people are building on top of it. People are using coding agents to quickly integrate with Google Search Console, Google Ads and building their own AI SEO and reporting features. Many were already building their own custom solutions around…

  Dev tools · Jul 2026 · openseo.so