AI · alternatives · 2026
24 alternatives to Shoehorn, a library to quantize an LLM to fit your Mac's VRAM
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right now I am running Qwen3-30B-A3B on my 24gb unified memory m4 MacBook Pro at 50 tok/sec and this should definitely not be working for such a large model on my middling hardware. Things are detailed in the README to get up and running and DESIGN.md has details on all the choices and such made along the way.
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Prices are shown for the 1 we have checked on their own sites, 1 of them free or with a free tier.
- 1
I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift. It ships with auto-mode, which makes a good tradeoff between memory usage and speed. I'll be implementing and porting the MTP module for speculative decoding next Local models really are the future of computing!
Sep 2026 · github.com · its alternatives →
- 2
The video demo runs a 7b Model on a normal gaming GPU. I think it already works quite well (accounting for the limited hardware power). :)
2024 · github.com · its alternatives →
- 3
Working on Mac, Linux, and Windows now. I include a simple GUI to find new models and get things built and set up. It is working quite well across a few models for me. The GitHub README and DESIGN.md files go into detail of the how/why and it's working remarkably well so far. https://github.com/notactuallytreyanastasio/shoehorn
Aug 2026 · notactuallytreyanastasio.github.io · its alternatives →
- 4
Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
2025 · caniusellm.com · its alternatives →
- 5
FreeUpMyMac▲110Hey HN Ex YC/Google engineer turned indie maker here - excited to share something I just shipped. The System Data in my Mac was taking up 300GB and it took me months to figure out how to clear the old cache, logs, junk and more. And CleanMyMac is a subscription - I refuse to pay subscription fees for Mac apps that run locally. So I built FreeUpMyMac an offline, lifetime Mac cleaner that clear junk with 1 click. And helps you visualize your storage with TreeMap/Sunburst. Hope it’s helpful to you as it is for me. Would love your feedback Tunde
Jan 2026 · freeupmymac.com · its alternatives →
- 6
I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edge devices like the Jetson Orin super nano kit which have unified memory. I haven't benched much, but plan to do so more over time and see if I can make a real use of it as I run local LLMs. Let me know if it makes sense :P (I of course vibed this idea)
Jun 2026 · github.com · its alternatives →
- 7IM
2023 · vram.asmirnov.xyz · its alternatives →
- 8WM
Pick a Mac and a model, get the memory it needs and the tokens/sec to expect. Covers M1 to M6, every unified memory size, Q4 to FP16. Formula fitted and validated on our own measurements on two machines.
7d ago · macyou.co · its alternatives →
- 9
- 10TF
I’d originally launched my app: Private LLM[1][2] on HN around 10 months ago, with a single RedPajama Chat 3B model. The app has come a long way since then. About a month ago, I added support for 4-bit OmniQuant quantized Mixtral 8x7B Instruct model, and it seems to outperform Q4 models at inference speed and Q8 models at text generation quality, while consuming only about 24GB of RAM[3] at 8k context length. The trick is: a) to use a better quantization algorithm and b) to use unquantized embeddings and the MoE gates (the overhead is quite small). Other notable features include many more…
2024 · its alternatives →
- 11
It's a chrome extension that automatically loads the specs from the Hugging Face model card into the calculation. > To test it, install the extension (no registration/key needed) and navigate to a HF model page. Then click the "VRAM" icon on the top right to open the sidepanel. You can specify quantization, batch size, sequence length, etc. Works for inference & fine-tuning. If it does not fit on the specified GPUs, it gives you an advise on how to still run it (e.g. lowering precision). It is inspired at my work, where we were constantly exporting metrics from HF to estimate required…
2025 · chromewebstore.google.com · its alternatives →
- 12
Democratisation of local AI is key. I've been working on pushing the limits of commercial hardware, squeezing any extra bit possible. My Scientific Agentic AI hareness helped me to reallocate every single bit of it. I rewrote the Kernel, I went down the CUDA rabbit hole until I have been able to explain any bit and any ms of computational power involved in the process pushing the Qwen 30B-A3B from 8 tok7s to 19 tok/s with llama.cpp up to 22.2 tok/s with my project and 109 tok/s on not novel content and speeding up the prefill by 5-9X
Jul 2026 · github.com · its alternatives →
- 13
LLM·RAG·VLM·아바타 워크로드를 GPU·CPU·RAM·스토리지·네트워크 구성과 경제형·권장형·확장형 견적 3안으로 변환하는 오픈소스 AI 인프라 산정 도구
Aug 2026 · jaeseok614.github.io · its alternatives →
- 14
If I have this hardware, Can I run that LLM model ?
2025 · can-i-run-this-llm-blue.vercel.app · its alternatives →
- 15
Abstract page for arXiv paper 2607.23806: A Frozen 12B Beats Frontier Models on Verified Work: 100% Accuracy, 0 Tokens, Bit-Exact, Forever
Jul 2026 · arxiv.org · its alternatives →
- 16
Hey HN! I made a completely open sourced alternative to Weights and Biases with (insert cringe) blazingly fast performance (yes we use rust and clickhouse) Weights and Biases is super unperformant, their logger blocks user code... logging should not be blocking, yet they got away with it. We do the right thing by being non blocking. Would love any thoughts / feedbacks / roasts etc
2025 · github.com · its alternatives →
- 17
Hi HN, I built OpenGraviton, an open-source AI inference engine that pushes the limits of running extremely large LLMs on consumer hardware. By combining 1.58-bit ternary quantization, dynamic sparsity with Top-K pruning and MoE routing, and mmap-based layer streaming, OpenGraviton can run models far larger than your system RAM—even on a Mac Mini. Early benchmarks: TinyLlama-1.1B drops from ~2GB (FP16) to ~0.24GB with ternary quantization. At 140B scale, models that normally require ~280GB fit within ~35GB packed. Optimized for Apple Silicon with Metal + C++ tensor unpacking, plus…
Mar 2026 · github.com · its alternatives →
- 18
Hey yall, I built this for my makerspace. We wanted to let each other borrow things, but also didn't want to put up a big shiny sign saying "hey everyone, my garage has a super fancy oscilloscope just sitting there" and also didn't want to necessarily always need to say yes to everything. However, when I show it to people, they've been suggesting using it for other things like board games, or wedding equipment, etc. I'm still trying to find PMF, so please let me know what you think! For the stack, it's all Elixir + LiveView which is really nice, and then Fly.io, Fly's MPG, Tigris Data,…
Aug 2026 · toolpool.garden · its alternatives →
- 19
After seeing a cool demo of a hack on Twitter, I built a cross platform version of it that works well and uses streaming. From anywhere on Mac and Linux, trigger Ollama and optionally feed it your clipboard. I built it yesterday and it's already very useful to me. I'm pretty excited about it and wanted to share!
2024 · github.com · its alternatives →
- 20
With the latest launch from Google I've added support for Gemma 3 270M, the speed for local LLM to TTS token time is incredible! This is an heavy obvious work in progress - any contributions or tips would be welcome. The idea is to have a fast moving edge model playground, and maybe have some utility (like the e reader) on the side.
2025 · github.com · its alternatives →
- 21
Hello all, a couple of moons ago I ported karpathy's llama2.c code to run inference on the TinyStories 260K & 15M checkpoints on the on the PS Vita with the ability to download/delete the models on device. Runs showed that the 260K model ran at ~120 tok/s and at 15M ran at 1.8 tok/s, which probably could be a bit higher if it weren't a single threaded application. Had fun working on it as a weekend project, check it out for yourselves: https://github.com/callbacked/psvita-llm
2025 · github.com · its alternatives →
- 22IB
Hey HN, I've been working on something cool that I wanted to share with you all. It's called Viewpoint, an analytics tool for LLMs like OpenAI, Anthropic models, and Gemini. The idea came from the constant flood of new LLM models and the need to figure out which ones work best for my projects without breaking the bank. With viewpoint, I can track token usage, costs, latency(WIP), and traffic over time, making it easier to compare different models and see which ones perform best and save money. The tool works asynchronously, so it doesn't add any latency to your LLM requests, and you have…
2024 · viewpointhq.com · its alternatives →
- 23
I made my first macOS utility app that ships with a bundled Gemma 4 model, specifically the Gemma E4B one. It made my app DMG have 5.3 GB in size, but I think it is a small size for the power that this free local model can provide. It runs fine on CPU, but can also run on Apple Silicon GPU, although I did not notice any performance improvements with GPU (tested on a M5 chip). I think these local lightweight and multimodal models will open multiple possibilities for new software tools where privacy is essential.
May 2026 · snapname.app · its alternatives →
- 24IB
I had 14,000 photos sitting on a drive and wanted an excuse to play with local vision models and Elixir/Phoenix. I originally tried to get LLaVA to tell me if a photo was 'good' or matched my style, but quickly learned that LLMs have terrible taste. I ended up demoting the LLM to just extract metadata, and built a custom CLIP/Ridge Regression pipeline to actually learn my preferences based on how I rate things. The stack is Phoenix/Oban on the orchestrator side, and Python/FastAPI/Instructor for the AI workers. Happy to answer any questions about the architecture,…
Apr 2026 · qwelian.com · its alternatives →
Also compare
- Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s alternatives
- I've built a locally running Perplexity clone alternatives
- Shoehorn – Quantize any model down to run on your machine alternatives
- I built a tool to check if your computer can run LLMs locally alternatives
- FreeUpMyMac alternatives
- KV-psi, using Linux PSI to to trim an LLM KV cache alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →