AI · alternatives · 2026
24 alternatives to Sipp – Run small local LLMs in browser 3x faster
Hi HN! Sipp is an open-source AI inference library for running local models in browsers with up to 3x faster decode speeds than alternative libraries. My background is in HCI (human-computer interaction) and graphics programming. Me along with my co-founder have been experimenting and thinking a lot about what the next user experience will look like when tokens are commodified to the point of being essentially “free.” A motivation for us was to try to move beyond the chat app and information retrieval use cases that are dominant now, and figure out how AI could instead act as a continuous…
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
2024 · asciinema.org · its alternatives →
- 2
Opper AI▲229Hey HN! After the Car Wash Test post got quite a big discussion going (400+ comments, https://news.ycombinator.com/item?id=47128138), I spent the past few weeks building a tool so anyone can run these kinds of questions and get structured results. No signup and free to use. You type a question, define answer options, pick up to 50 models at a time from a pool of 200+, and they all answer independently under identical conditions. No system prompt, structured output, same setup for every model. You can also run a debate round where models see each other's reasoning and get a…
Jul 2026 · opper.ai · its alternatives →
- 3
OpenWispr▲190100% local open source AI speech-to-text model
2025 · openwhispr.com · its alternatives →
- 4
We wrote our inference engine on Rust, it is faster than llama cpp in all of the use cases. Your feedback is very welcomed. Written from scratch with idea that you can add support of any kernel and platform.
2025 · github.com · its alternatives →
- 5
Hi HN, OpenAI recently released a model for automatic speech recognition called Whisper [0]. I decided to reimplement the inference of the model from scratch using C/C++. To achieve this I implemented a minimalistic tensor library in C and ported the high-level architecture of the model in C++. The entire code is less than 8000 lines of code and is contained in just 2 source files without any third-party dependencies. The Github project is here: https://github.com/ggerganov/whisper.cpp With this implementation I can very easily build and run the model - “make…
2022 · github.com · its alternatives →
- 6
Hello HN, I don't post on here much, but wanted to get some eyes on a new project I'm just launching. I think we definitely need one more AI code agent.. I'm a long-term C++ dev, and over 30+ years I've created some successful audio dev tools (JUCE, the Tracktion DAW, the Cmajor DSP language). All of these came from me getting annoyed with something I had to use, and deciding to have a go at my own take on whatever it was. So Juggler is my attempt at an AI code agent, after spending too many hours loving what the models could do, but hating the CLI experience, and having some opinions of…
Jul 2026 · github.com · its alternatives →
- 7
Context Gateway▲220We built an open-source proxy that sits between coding agents (Claude Code, OpenClaw, etc.) and the LLM, compressing tool outputs before they enter the context window. Demo: https://www.youtube.com/watch?v=-vFZ6MPrwjw#t=9s. Motivation: Agents are terrible at managing context. A single file read or grep can dump thousands of tokens into the window, most of it noise. This isn't just expensive — it actively degrades quality. Long-context benchmarks consistently show steep accuracy drops as context grows (OpenAI's GPT-5.4 eval goes from 97.2% at 32k to 36.6% at 1M…
Mar 2026 · github.com · its alternatives →
- 8
Sipcode▲148Keep Claude Code's context clean for sharper answers
Jun 2026 · anuj7411.github.io · its alternatives →
- 9
We built a model router that plugs into coding agents (e.g. Claude Code, Codex, Cursor, etc.) and intelligently sends requests to the best model to serve them. Here's a quick demo of running it locally: https://www.youtube.com/watch?v=isKhAyivtfM. At Weave, we write most of our code with AI, and it's been getting more expensive. This came to a head when Opus 4.7 was released and, thanks to its tokenizer changes, our costs shot up. We knew we didn't need Opus for everything but we didn't want to lose out on the intelligence for the cases where you really need it. So we decided…
Jun 2026 · github.com · its alternatives →
- 10
Your AI agent is burning 6x more tokens than it needs to just to browse the web. We built OpenBrowser MCP to fix that. Most browser MCPs give the LLM dozens of tools: click, scroll, type, extract, navigate. Each call dumps the entire page accessibility tree into the context window. One Wikipedia page? 124K+ tokens. Every. Single. Call. OpenBrowser works differently. It exposes one tool. Your agent writes Python code, and OpenBrowser executes it in a persistent runtime with full browser access. The agent controls what comes back. No bloated page dumps. No wasted tokens. Just the data your…
Apr 2026 · openbrowser.me · its alternatives →
- 11
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
May 2026 · github.com · its alternatives →
- 12
Hi HN! Last year the project I launched here got a lot of good feedback on creating speech to speech AI on the ESP32. Recently I revamped the whole stack, iterated on that feedback and made our project fully open-source—all of the client, hardware, firmware code. This Github repo turns an ESP32-S3 into a realtime AI speech companion using the OpenAI Realtime API, Arduino WebSockets, Deno Edge Functions, and a full-stack web interface. You can talk to your own custom AI character, and it responds instantly. I couldn't find a resource that helped set up a reliable, secure websocket (WSS) AI…
2025 · github.com · its alternatives →
- 13
Cai▲179Hi HN, Cai runs actions on your selection via ⌥C, pick an action (AI, shell, GitHub issue, Linear ticket), done. Bundled local model (MLX Community), create custom actions with prompts, scripts and more. Free, MIT. Built after my Raycast extension sat in review queue for 3+ months. Wanted no typing, no app switching, no cloud. Repo: https://github.com/cai-layer/cai Demo: https://youtu.be/LH2MYNQ_wsk
Apr 2026 · getcai.app · its alternatives →
- 14
SIP library for writing fast SIP services in GO. Contribute to emiago/sipgo development by creating an account on GitHub.
2023 · github.com · its alternatives →
- 15
Voxos is an open-source desktop voice assistant that aims to put Clippy to shame while supporting new desktop workflows powered by LLMs. Tired of copy and pasting ChatGPT responses between your web browser and IDE? Does your copilot not quite do what you need it to do? I invite you to give Voxos a try and maybe even become a contributor!
2024 · gitlab.com · its alternatives →
- 16
This is an open‑source Model Context Protocol (MCP) server that gives any LLM a sense of the passage of time. Most MCP demos wire LLMs to external data stores. That’s useful, but MCP is also a chance to give models perception — extra senses beyond the prompt text. Six functions (`current_datetime`, `time_difference`, `timestamp_context`, etc.) give Claude/GPT real temporal awareness: It can spot pauses, reason about rhythms, and even label a chat’s “three‑act structure”. Runs locally in <60 s (Python) or via a hosted demo. If time works, what else could we surface? - Location /…
2025 · github.com · its alternatives →
- 17
Check out this impressive project that enables running LLMs entirely in the browser using WebGPU. Key features: - Zero token costs, no cloud infrastructure required - Complete data privacy through local processing - Simple 3-line code integration - Built on MLC and Transformer.js The benchmarks show smaller models can effectively handle many common tasks. Currently the project roadmap includes: - No-code AI pipeline builder - Browser-based RAG for document chat - Analytics/logging - Model fine-tuning interface
2025 · github.com · its alternatives →
- 18
Free MCP for security AI: live BGP, DNS, threat graph
May 2026 · whisper.security · its alternatives →
- 19AM
Hi all, I’m hacking on new features for the ClickHouse native client and wanted the same “just call the model” ergonomics JavaScript and Python now enjoy. It didn’t exist for modern C++, so I wrote one. ai‑sdk‑cpp (Apache‑2.0) gives you: - Unified calls to OpenAI (GPT‑4o) and Anthropic (Claude 3.5) with a single C++20 API. - Streaming, multi‑turn chat, error handling—all std::optional/std::variant, no macros. - Tool calling (function‑calling) so the model can hit real APIs; sync or async, runs in parallel. The tricky bit: C++ still lacks real reflection, so mapping plain functions →…
2025 · its alternatives →
- 20TF
I’d originally launched my app: Private LLM[1][2] on HN around 10 months ago, with a single RedPajama Chat 3B model. The app has come a long way since then. About a month ago, I added support for 4-bit OmniQuant quantized Mixtral 8x7B Instruct model, and it seems to outperform Q4 models at inference speed and Q8 models at text generation quality, while consuming only about 24GB of RAM[3] at 8k context length. The trick is: a) to use a better quantization algorithm and b) to use unquantized embeddings and the MoE gates (the overhead is quite small). Other notable features include many more…
2024 · its alternatives →
- 21AT
I recently built a small open-source tool to benchmark different LLM API endpoints — including OpenAI, Claude, and self-hosted models (like llama.cpp). It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including…
2025 · llmapitest.com · its alternatives →
- 22OS
Large Language Models (LLMs) are powerful, but they’re limited by fixed context windows and outdated knowledge. What if your AI could access live search, structured data extraction, OCR, and more—all through a standardized interface? We built the JigsawStack MCP Server, an open-source implementation of the Model Context Protocol (MCP) that lets any AI model call external tools effortlessly. Here’s what it unlocks: - Web Search & Scraping: Fetch live information and extract structured data from web pages. - OCR & Structured Data Extraction: Process images, receipts, invoices, and handwritten…
2025 · its alternatives →
- 23
Hey HN, We’re two developers (co-founders) with a team of 20 who got tired of spending hours reviewing PRs, so we built Infinitcode.ai, an AI-powered code reviewer that: - *Summarizes PRs in plain English*: No more deciphering 1,000-line diff jungles - *Catches more than bugs*: Security holes, performance pitfalls, code smells, even typos (yes, we’ll flag “vurnerabilities” and vulnerabilities) - *Zero onboarding*: Works instantly—no “let me learn your codebase for weeks” nonsense. Why we’re posting: We’re in alpha and need brutal honesty. Roast our tool, mock our UI, or tell us why AI will…
2025 · infinitcode.ai · its alternatives →
- 24
Local In-Browser LLM inference for private, fast, and secure chat - no server required.
2025 · private-ai-chat.vercel.app · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →