Alternatives
Products that do what Cactus Hybrid: We taught Gemma 4 to know when it's wrong does
Hey HN, Henry & Roman here from Cactus. A small, on-device model is fast and private, but sometimes wrong, but frontier models are getting expensive pretty fast. So, we post-trained Gemma 4 E2B post-trained to know when it's wrong. Every response comes with a confidence score between 0 and 1. Developers can accept the on-device when it's high, hand off to a bigger cloud model when it's low. By routing only 15-35% of queries to Gemini 3.1 Flash-Lite, Gemma-4-E2B matches Gemini 3.1 Flash-Lite on most benchmarks. - ChartQA: 15-20% - LibriSpeech: 25-30% - MMBench, GigaSpeech, MMAU: 30-35% -…
- 1
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
27d ago · cactuscompute.com
- 2OS
Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal. I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory. The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are…
Jul 2026 · github.com
- 3
- 4
- 5
- 6
- 7
- 8IM
I made my first macOS utility app that ships with a bundled Gemma 4 model, specifically the Gemma E4B one. It made my app DMG have 5.3 GB in size, but I think it is a small size for the power that this free local model can provide. It runs fine on CPU, but can also run on Apple Silicon GPU, although I did not notice any performance improvements with GPU (tested on a M5 chip). I think these local lightweight and multimodal models will open multiple possibilities for new software tools where privacy is essential.
May 2026 · snapname.app
- 9
- 10
Gemini 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber
Jul 2026 · blog.google
- 11
- 12FA
Sharing instructions on finetuning Gemma2b for a codegen usecase!
2024 · github.com
- 13
- 14NT
With the latest launch from Google I've added support for Gemma 3 270M, the speed for local LLM to TTS token time is incredible! This is an heavy obvious work in progress - any contributions or tips would be welcome. The idea is to have a fast moving edge model playground, and maybe have some utility (like the e reader) on the side.
2025 · github.com
- 15IM
Hey HN! Thank you for all the support and feedback on my original submission 2 months ago. I've been improving the backend using a MCTS/AlphaZero approach and it's currently producing much better results. My long term goal is to allow users to manage multiple projects, deployed autonomously, both from scratch and by making continual updates all prompted with natural language. The cost of each project has been lowered to $9 as performance with smaller models has improved (I migrated from Claude-3-Opus to gemini-1.5-flash). Thanks for checking it out!
2024 · saas-quick.com
- 16GV
Hey HN, I just updated my project that compares some LLMs. It uses your prompt for all the models and runs at the same time. You can see the results being generated in real-time and decide what's the best for your use case. I'm open to any suggestions and feedback. Thanks!
2024 · geminivsgpt.com
- 17SC
Unlike LoRA and its variants, which inject trainable parameters directly into the weights of the Transformer, requiring tight coupling with the backbone. ShadowPEFT instead enhances the frozen large base model by adding a lightweight, centralized, pretrainable, and detachable Shadow network. This shadow network operates in parallel with the base model, delivering learned corrections to each decoder layer. Because the shadow module is architecturally decoupled from the backbone, it can be independently trained, stored, and deployed, benefiting edge computing scenarios and edge-cloud…
Apr 2026 · github.com
- 18JN
We’ve been experimenting with how far a tiny model can go when it’s good at calling external tools - and have just released Jan-nano, a 4 B model trained for MCP. Jan-nano: - tops DeepSeek-V3-671B on MCP tool-use (SimpleQA 80.7%) - handles live web search and multi-step deep research - runs fully on-device (≈4GB VRAM) Tech notes - Base: Qwen3-4B - Fine-tuning: DAPO - We're going to release the full technical report soon Links - Demo tweet: https://x.com/menloresearch/status/1934809407604576559 - Model + GGUF:…
2025 · twitter.com
- 19AB
Best to just try it :) The tl:dr is it helps you to r The summaries with references to parts of the page are useful now - and the "text zoom" tool a glimpse of where we can go Works with local Ollama and the usual suspects: Mistral/OpenAI/Gemini Obviously could be better - and hopefully will be :)
2024 · socontextual.com
- 20AC
Hi everyone, I've been working on a CLI tool that can help to easily run any model in claude, Codex, Gemini, Pi, and OpenCode. It's also an API keys manager, supports multiple providers or OpenAI/Claude/Gemini accounts. You can add openrouter, poe, Vercel AI gateways etc. It has a built-in provider that is free to all, which is using Deepseek-V4, no login or API key required, add your own when you're ready. After installation you can try claude instantly (No config, no login): aivo claude Hope it's useful to someone.
Apr 2026 · getaivo.dev
- 21HT
Fun little project, had Gemini 2.5 Pro summarize HN's top 30 each hour, both the stories and comment sections. Pretty impressed with Gemini 2.5. It's probably the first model other than Claude 3.7 Sonnet where I actually find the output readable. I normally use 3.7 Sonnet for coding, but used Gemini for the codegen on this one as well. Was pretty impressed! Using Cursor, it seemed to instruction-follow better than Claude generally does, and remain lucid during very long agent sessions. Thanks for your feedback!
2025 · tinysums.ai
- 22NT
I built a CLI tool that turns codebases and PRs into diagrams so you can quickly understand how things fit together. Originally made it because I couldn't follow my own AI-generated repos. Just shipped a big update: - Switched from D2 to Mermaid for rendering - Tree-sitter AST parsing + agentic flow instead of raw LLM calls. ~50x faster. - Works on any GitHub repo or PR, not just local - Dropped the web frontend, it's just a CLI now - Published as a pip package Still a ton to improve and I'm building fast. Feedback, issues, PRs all welcome.
Feb 2026 · github.com
- 23AO
Hey HN, My workflow for any complex queries is to ask it in multiple AI chats (Gemini, Claude, o3,..) in parallel and then continue the conversation with the chat response that I found the most useful. I built a simple open source app that queries 10+ AI models at once and summarizes their answers with a selected combiner AI model. There's a GIF in the github repo that shows it in action. You can try it on your local machine: https://github.com/Nexarithm/multi_model_chat If you are interested, I also made a detailed blog post on technical details, feature of the personal…
2025 · github.com
- 24CT
I had been looking to try <500M parameter language models but you wouldn't find an API to try them anywhere, so I built this cloudflare hosted static website that hosts weights and built an inference runtime for these models that uses WebGPU and runs inference from your browser. These are only so useful in a multi-turn conversation but it's still interesting to see what you can pack in a <250mb model. I tried using ONNX versions earlier, but there were too many quirks of using them with language models and the TPS wasn't too impressive. Inspired by svenflow/webgpu-gemma, I put my codex…
May 2026 · chonklm.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →