Alternatives
Products that do what Nabu (TTS Reader and LLM Playground on Android) does
With the latest launch from Google I've added support for Gemma 3 270M, the speed for local LLM to TTS token time is incredible! This is an heavy obvious work in progress - any contributions or tips would be welcome. The idea is to have a fast moving edge model playground, and maybe have some utility (like the e reader) on the side.
- 1
- 2IB
After fine-tuning GPT for a personal project, I realized how tedious it is to write plain text in a massive JSON file. That's why I built this app for my own use, and I want to see if others could benefit from a tool like this as well ;)
2024 · finetuna-ui.com
- 3ML
Time to first token is 39% faster Agent wall times decrease by 46% No swaps Tracks your resource usage in real-time and adjusts how the model runs so that it works perfectly on your device. Implements KV cache sizing, prefix caching, live RAM pressure management, context trimming, KV quantization, and more. Built a ton of features
Jun 2026 · autotunellm.com
- 4
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right now I am running Qwen3-30B-A3B on my 24gb unified memory m4 MacBook Pro at 50 tok/sec and this should definitely not be working for such a large model on my middling hardware. Things are detailed in the README to get up and running and DESIGN.md has details on all the choices and such made along the way.
23d ago · github.com
- 5PU
After seeing a cool demo of a hack on Twitter, I built a cross platform version of it that works well and uses streaming. From anywhere on Mac and Linux, trigger Ollama and optionally feed it your clipboard. I built it yesterday and it's already very useful to me. I'm pretty excited about it and wanted to share!
2024 · github.com
- 6IB
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com
- 7KP
I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edge devices like the Jetson Orin super nano kit which have unified memory. I haven't benched much, but plan to do so more over time and see if I can make a real use of it as I run local LLMs. Let me know if it makes sense :P (I of course vibed this idea)
Jun 2026 · github.com
- 8AO
I've built an airgapped Retrieval-Augmented Generation (RAG) system for question-answering on documents, running entirely offline with local inference. Using Llama 3, Mistral, and Gemini, this setup allows secure, private NLP on your own machine. Perfect for researchers, data scientists, and developers who need to process sensitive data without cloud dependencies. Built with Llama C++, LangChain, and Streamlit, it supports quantized models and provides a sleek UI for document processing. Check it out, contribute, or suggest new features!
2024 · github.com
- 9AL
Hello all, a couple of moons ago I ported karpathy's llama2.c code to run inference on the TinyStories 260K & 15M checkpoints on the on the PS Vita with the ability to download/delete the models on device. Runs showed that the 260K model ran at ~120 tok/s and at 15M ran at 1.8 tok/s, which probably could be a bit higher if it weren't a single threaded application. Had fun working on it as a weekend project, check it out for yourselves: https://github.com/callbacked/psvita-llm
2025 · github.com
- 10IM
I made my first macOS utility app that ships with a bundled Gemma 4 model, specifically the Gemma E4B one. It made my app DMG have 5.3 GB in size, but I think it is a small size for the power that this free local model can provide. It runs fine on CPU, but can also run on Apple Silicon GPU, although I did not notice any performance improvements with GPU (tested on a M5 chip). I think these local lightweight and multimodal models will open multiple possibilities for new software tools where privacy is essential.
May 2026 · snapname.app
- 11IB
Hey HN, I built a website where you can train Llama 3.1 8b & 70b (4bit) on your data. I use unsloth in the backend and the training is done on H100s which I rent programmatically from Runpod. I'd love some feedback. If you would be interested in using it feel free to book a chat with me: cal.com/hamada/tunellama-intro Happy to give you free credits :) P.S. I'm also looking for a co-founder as I have big plans for this.
2024 · tunellama.com
- 12IB
hey hn, I built an open-source Perplexity clone that can run local LLMs and cloud LLMs. It's fully self-hostable through Docker and uses ollama to support local LLMs. The demo video in the repository shows me running it locally with llama3 on my M1 Macbook Pro. I'm open to any suggestions or feedback, thanks!
2024 · github.com
- 13AU
Hi HN, I was once given the advice: Don't waste expensive frontier model credits (GPT/Claude/etc.) on bulk work. Send the boring, repetitive, high-volume jobs to a smaller model, and save the expensive prompts for when you actually need frontier-level reasoning. I complained and told my manager that I shouldnt have to think about using certain models for certain coding tasks, and that one model should handle everything. Well, here we are anyway. If anyone needs a place to absolutely abuse an LLM with high-volume tasks, come beat ours up at https://yolo-auto.com. Here are…
Jul 2026 · yolo-auto.com
- 14IM
Try it at: ssh krnl.duetbrowser.com No sign up, totally free - no download. Let me know what you think. Prior art exists (brow.sh) but I wasn't satisfied with their graphical take. I wanted the raw, old skool terminal days of the early internet - BBS, and old green phosphor library comptuers - but running the modern web, with all the bells and jingle jangle whistles (JavaScript, etc). No funding, no VC, just 100% raw terminal fun. Not sure how long the demo will last, but try it out.
Jul 2026
- 15LL
2023 · github.com
- 16RM
Apr 2026 · github.com
- 17GV
Hey HN, I just updated my project that compares some LLMs. It uses your prompt for all the models and runs at the same time. You can see the results being generated in real-time and decide what's the best for your use case. I'm open to any suggestions and feedback. Thanks!
2024 · geminivsgpt.com
- 18LS
I kept running into this annoying problem: I’d remember a really useful answer, but not where it was. ChatGPT? Claude? Gemini? No idea. So I’d end up digging through all of them or just rewriting the prompt. Built this to fix that. It’s a Chrome extension that indexes chats locally and lets you search across them all in one place. Once it’s indexed, search is basically instant. Still early. UIs change and break things sometimes, so it’s a bit fragile in places. Curious if other people have the same issue or if it’s just me jumping between tools too much.
Apr 2026 · chromewebstore.google.com
- 19AD
Hi all, I threw together a small prototype I am calling “Notepad.ai”. A new take on UIs for interacting with LLMs. While I enjoy using LLM’s in the chat format I wanted to see what it would be like to do it in a more long form style. It let’s you write in a pretty free form, much like Window’s Notepad, but you can choose to hit ctrl+[ to analyze the text with a preset prompt of your choosing. It has a few other small features. It’s WIP and very experimental. I would appreciate any feedback or thoughts. Video: https://youtu.be/ntdlgFmSxQY Live Demo:…
2024 · github.com
- 20AB
All LLM user interfaces I've seen so far are somewhat frustrating: * ChatGPT web requires a lot of copy-paste, it rewrites whole document even if you need to update a part of it, etc. * Github Copilot completions are rather unreliable and do not leave an option to specify what you want; Copilot's chat sidebar is little more than ChatGPT integrated into the IDE * Google Docs have right UI for non-code text, but they use really dumb model (not Gemini 1.5 Pro). Also won't work for code. So... I wrote a Emacs Lisp function which calls LLM with contents of the buffer to generate text according to…
2024 · x.com
- 21IM
Hi, Sometimes I want to wrap an existing website into an Android app quickly so I made a service that does just that https://html2apk.com/ If you find it useful I'll be happy to hear about that, if you have any other comments/suggestions I'll be happy to hear about that too. 10x, Gilad
2019
- 22IB
I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things. The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture,…
Apr 2026 · qwelian.com
- 23GR
Hey folks, As much as we love GPT-4, it's expensive and can be slow at times. That's why we built GPTCache - a semantic cache for autoregressive LMs - atop the vector database Milvus and SQLite. GPTCache provides several benefits: 1) reduced expenses due to minimizing the number of requests and tokens sent to the LLM service 2) enhanced performance by fetching cached query results directly 3) improved scalability and availability by avoiding rate limits, and 4) a flexible development environment that allows developers to verify their application's features without connecting to LLM APIs or…
2023 · github.com
- 24IB
I built Read Fast after realising I was saving hundreds of things to read and reading almost none of them. It automatically turns saved content into a scheduled briefing and sends it to my inbox at whatever time I choose. New version since last post actually allows you to bulk upload URLs. So you can just dump links here and Read Fast will scrape and brief for you. Great for actually reading the things I want to. Curious if that resonates with people?
Dec 2025 · read-fast.replit.app
Ranked by how close each launch is in meaning, then by votes. Refine with a description →