Shoehorn, a library to quantize an LLM to fit your Mac's VRAM
I made this after seeing someone posit the idea online yesterday over lunch then spent some time refining it. So far it's pretty impressive IMO! Right now I am running Qwen3-30B-A3B on my 24gb unified memory m4 MacBook Pro at 50 tok/sec and this should definitely not be working for such a large model on my middling hardware. Things are detailed in the README to get up and running and DESIGN.md has details on all the choices and such made along the way.
Shoehorn, a library to quantize an LLM to fit your Mac's VRAM launched on August 14, 2026 with 6 votes, #1,590 of 23,274 launches that month and more than 75% of that year's launches.
What it does
Contribute to notactuallytreyanastasio/shoehorn development by creating an account on GitHub.
Does a similar job
all alternatives →
Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/sSep 2026 · github.com · ▲240I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift. It ships with auto-mode, which makes a good tradeoff between memory usage and speed. I'll be implementing and porting the MTP module for speculative decoding next Local models really are the future of computing!
Shoehorn – Quantize any model down to run on your machineAug 2026 · notactuallytreyanastasio.github.io · ▲99Working on Mac, Linux, and Windows now. I include a simple GUI to find new models and get things built and set up. It is working quite well across a few models for me. The GitHub README and DESIGN.md files go into detail of the how/why and it's working remarkably well so far. https://github.com/notactuallytreyanastasio/shoehorn
I built a tool to check if your computer can run LLMs locally2025 · caniusellm.com · ▲8Built a simple web app that tells you which open-source LLMs will work on your hardware. It auto-detects your specs, shows compatible models from Hugging Face, gives realistic performance estimates (tokens/sec), and recommends quantization settings. You can also manually input specs to see "what if I upgraded my RAM?" Made this after wasting time downloading giant models only to find they crawled on my hardware. Hope it saves you some frustration!
FreeUpMyMacJan 2026 · freeupmymac.com · ▲110Hey HN Ex YC/Google engineer turned indie maker here - excited to share something I just shipped. The System Data in my Mac was taking up 300GB and it took me months to figure out how to clear the old cache, logs, junk and more. And CleanMyMac is a subscription - I refuse to pay subscription fees for Mac apps that run locally. So I built FreeUpMyMac an offline, lifetime Mac cleaner that clear junk with 1 click. And helps you visualize your storage with TreeMap/Sunburst. Hope it’s helpful to you as it is for me. Would love your feedback Tunde
KV-psi, using Linux PSI to to trim an LLM KV cacheJun 2026 · github.com · ▲8I thought it'd be interesting to use Linux PSI (Pressure Stall Information) for an LLM runtime to trim the KV cache. This is mainly useful imo for edge devices like the Jetson Orin super nano kit which have unified memory. I haven't benched much, but plan to do so more over time and see if I can make a real use of it as I run local LLMs. Let me know if it makes sense :P (I of course vibed this idea)
- WMWhich Mac runs which LLM, and how fast7d ago · macyou.co · ▲10
Pick a Mac and a model, get the memory it needs and the tokens/sec to expect. Covers M1 to M6, every unified memory size, Q4 to FP16. Formula fitted and validated on our own measurements on two machines.
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗
Launched alongside, August 2026
the whole month →- TL
Life & fun · Aug 2026 · louisabraham.github.io
Clipto MCP▲655Hey HN, We recently built Clipto. It’s a tool that lets you search over terabytes of video, audio, and images on your computer, without relying on the cloud. Motivation: we probably all had this similar experience, we know a moment exists in a video or audio, but finding it takes hours scrubbing the timeline. You can send all the media to process in the cloud, but it’s slow, expensive and raises privacy concerns. So we decided to build our own on-device media search engine. How it works (high level): 1. We ingest video, audio, image; normalize formats via ffmpeg; run content analysis to…
Work · Aug 2026 · clipto.com
Hello HN! I found that picking out plausible but diverse skin tones for my digital art and game development projects was kind of difficult, and I got curious about if there was a way to define a color space that made it easy. I've built a color picker and procedural generation algorithm based on the space as well as a bunch of other fun js features and demos throughout the page that use the equations. If you find it interesting, I have lots of explanations of how I built it and what properties the space has. The methodology might be a bit shaky, but hopefully the result is as helpful for…
Life & fun · Aug 2026 · toneyalexander.github.io
Astute▲606Automate your B2B brand going viral, with new media creators
AI · Aug 2026 · company-app.joinastute.com