Alternatives
Products that do what Sipp – Run small local LLMs in browser 3x faster does
Hi HN! Sipp is an open-source AI inference library for running local models in browsers with up to 3x faster decode speeds than alternative libraries. My background is in HCI (human-computer interaction) and graphics programming. Me along with my co-founder have been experimenting and thinking a lot about what the next user experience will look like when tokens are commodified to the point of being essentially “free.” A motivation for us was to try to move beyond the chat app and information retrieval use cases that are dominant now, and figure out how AI could instead act as a continuous…
- 1
- 2
- 3TV
May 2026 · github.com
- 4JA
Hello HN, I don't post on here much, but wanted to get some eyes on a new project I'm just launching. I think we definitely need one more AI code agent.. I'm a long-term C++ dev, and over 30+ years I've created some successful audio dev tools (JUCE, the Tracktion DAW, the Cmajor DSP language). All of these came from me getting annoyed with something I had to use, and deciding to have a go at my own take on whatever it was. So Juggler is my attempt at an AI code agent, after spending too many hours loving what the models could do, but hating the CLI experience, and having some opinions of…
Jul 2026 · github.com
- 5SM
We built a model router that plugs into coding agents (e.g. Claude Code, Codex, Cursor, etc.) and intelligently sends requests to the best model to serve them. Here's a quick demo of running it locally: https://www.youtube.com/watch?v=isKhAyivtfM. At Weave, we write most of our code with AI, and it's been getting more expensive. This came to a head when Opus 4.7 was released and, thanks to its tokenizer changes, our costs shot up. We knew we didn't need Opus for everything but we didn't want to lose out on the intelligence for the cases where you really need it. So we decided…
Jun 2026 · github.com
- 6
- 7
- 8
- 9LT
Hi HN! I'm the author of mere.run a local first inference runtime built around an installable CLI. I believe that whenever possible we should use the stuff we already own (like our Mac laptops, decent machines gathering dust, our gaming PC) and the limited electrical power we have easy access to, like the socket in the wall next to most of us. We shouldn't have to send our data to the cloud hoping some T&C will prevent it from being used in a way that we'd regret. Most of the local AI solutions are technical, involved, and land a curious body in some package hell. People are optimizing for…
Jul 2026 · github.com
- 10BO
Read the full blogpost at https://rach.codes/blog/Introducing-Bhumi (click on reader to see the technical breakdown!) AI inference should be fast, but in practice it’s painfully slow. Inference bottlenecks slow down LLM-powered chatbots and AI workflows everywhere. I built Bhumi to fix that. Bhumi is a Python library designed for developers, yet its performance-critical core is implemented in Rust (via PyO3) for near-native speed. This hybrid approach delivers up to 2.5x faster response times across providers like OpenAI, Anthropic, and Gemini—without changing the…
2025 · bhumi.trilok.ai
- 11ZL
Zep is a long-term memory store designed for conversational AI applications built using modern LLMs. It handles the storage, summarization, embedding, indexing, and enrichment of chat histories, and offers developers a simple, low-latency API to this data. Chat history storage is an infrastructure challenge all developers and enterprises face as they look to move from prototypes to deploying conversational AI applications that provide rich and intimate experiences to users. Key features include long-term memory persistence, auto-summarization, vector search, auto-token counting, and Python…
2023
- 12AD
Hi all, I threw together a small prototype I am calling “Notepad.ai”. A new take on UIs for interacting with LLMs. While I enjoy using LLM’s in the chat format I wanted to see what it would be like to do it in a more long form style. It let’s you write in a pretty free form, much like Window’s Notepad, but you can choose to hit ctrl+[ to analyze the text with a preset prompt of your choosing. It has a few other small features. It’s WIP and very experimental. I would appreciate any feedback or thoughts. Video: https://youtu.be/ntdlgFmSxQY Live Demo:…
2024 · github.com
- 13TC
Hi HN, I'm thrilled to share with you think, a command-line tool that I developed while diving back into the programming scene. For the past few years, my focus had shifted from hands-on programming to managerial roles, my last one being at Affirm (I started Affirm's Polish office). However, I recently left my job to revisit my passion for coding and see if I could still write code that works. Not to mention that with what is currently going on in AI if you are a developer this probably the best possible time for a sabbatical in human history :) My first attempt was with LangChain, but I…
2023 · github.com
- 14RA
Hi there, looking for feedback on my new project "Featherless.AI" The idea is to allow users to run all the models on hugging face instantly. Via the OpenAI API compatible endpoint. Why? Because its a real chore to download models and spin up GPUs, especially if you want to test multiple models. Not to mention GPUs cost multiple dollars an hour to rent. And if we want more people to use open source AI, we got to make it easier for them to try and play with all of them. So what if instead of spinning up dedicated GPUs per model (which is what every provider is doing) We can startup a LLM…
2024 · featherless.ai
- 15MA
I've been working on training this small vision language model for the last month - excited to release the first prototype today! It is based on SigLIP (image encoder), Phi-1.5 (text model) and trained using the LLaVa-1.5 training dataset. It runs reasonably fast on CPU with ~8GB of RAM in full 32-bit precision. There's plenty of room to speed it up and reduce memory consumption by quantizing the model. I posted a video of it running on my M2 Macbook Air (on CPU not MPS, so performance should be comparable on other hardware) on Twitter to demonstrate inference speed:…
2023 · github.com
- 16
- 17MM
Hey HN! I’m Gui from deco (decocms.com). We’ve been using this tool internally as the foundation for a few customer AI platforms, and today we’re open-sourcing it as MCP Mesh. MCP is quickly becoming the standard for agentic systems, but… once you go past a couple servers it turns into the same problems for every team: - M×N config sprawl (every client wired to every server, each with its own JSON + ports + retries) - Token + tool bloat (dumping tool definitions into every prompt doesn’t scale) - Credentials + blast radius (tokens scattered across clients, hard to audit, hard to revoke) - No…
Dec 2025 · github.com
- 18
- 19LA
Hi HN, I’m excited to share LocoStudio (http://locostudio.ai), a local-first AI chat app for Mac that lets you chat with AI models both locally and in the cloud—all in one place. Why I built it: I wanted a private, fast, and flexible way to use the latest AI models without sending my data to the cloud. Most AI chat apps are either cloud-only (raising privacy concerns) or have clunky interfaces for local models. LocoStudio aims to make local AI chat as easy and powerful as using ChatGPT, but with your data staying on your device. What it does: - Private, local-first: Run open models…
2025 · locostudio.ai
- 20AR
Hi HN. I'm the founder of Phoenix Labs (ex TikTok, Applied AI) and we're open sourcing our internal tooling today which is like a toolchain / meta-harness for CLI agents useful for really scaling eng and creative work. We are a very small team who's building a very ambitious product so we had to find ways to squeeze every ounce of efficiency that we could get our hands on. Harness strengths of different models (Claude, GPTs) and CLI-harnesses (Claude Code, Codex), safe/robust browser integration to speed up UX/QA testing, teams cli to speed up security reviews and parallelize…
May 2026 · agents-cli.sh
- 21NL
Built this because I was tired of every AI tool shipping my data to someone else server n0x runs the full stack LLM inference via WebGPU, autonomous ReAct agents, RAG over your own docs, sandboxed Python execution via Pyodide all inside a single browser tab. No account No keys No backend Models download once, cache in IndexedDB permanently. Biggest challenge was context window budgeting for the agent loop and making the WASM vector search non-blocking. Happy to talk architecture. GitHub: https://github.com/ixchio/n0x | Live demo: https://n0x-three.vercel.app
Mar 2026 · n0xth.vercel.app
- 22
HAI 1.0: 1.2GB local AI. Beats GPT-5 at coding
Jul 2026 · github.com
- 23CA
Hi HN, I've been working with LLMs in production for a while both as a solo dev building apps for clients and working at an AI startup. The one thing that always was a pain was to pay OpenAI/Gemini/Anthropic a few dollars a month just for me to say "test" or have a CI runner validate some UI code. So I built this server called ChunkBack, that mocks the popular llm provider's functionality but allows you to type in a deterministic language: `SAY "cheese"` or `TOOLCALL "tool_name" {} "tool response"` I've had to work in some test environments and give good results for experimenting…
Nov 2025 · github.com
- 24IB
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →