OpenGraviton – Run 500B+ parameter models on a consumer Mac Mini
Hi HN, I built OpenGraviton, an open-source AI inference engine designed to push the limits of running extremely large models on consumer hardware. The system combines several techniques to drastically reduce memory and compute requirements: • 1.58-bit ternary quantization ({-1, 0, +1}) for ~10x compression • dynamic sparsity with Top-K pruning and MoE routing • mmap-based layer streaming to load weights directly from NVMe SSDs • speculative decoding to improve generation throughput These allow models far larger than system RAM to run locally. In early benchmarks, OpenGraviton reduced…
OpenGraviton – Run 500B+ parameter models on a consumer Mac Mini launched on March 7, 2026 with 13 votes, #1,024 of 23,380 launches that month and more than 90% of that year's launches.
In plain words
OpenGraviton is an open-source AI inference engine that enables running extremely large language models on consumer Mac hardware by combining ternary quantization, dynamic sparsity, layer streaming from NVMe storage, and speculative decoding. It reduces model memory requirements by up to 10 times, allowing 500 billion parameter models to run on machines with limited RAM. The system is optimized for Apple Silicon and targets developers and researchers who want to run large AI models locally without expensive server infrastructure.
written from the facts on this page · September 2026
From the sources
Graviton — open-source engine that runs 500B+ parameter LLMs locally on a Mac Mini. AI belongs to everyone, not corporations. Break free from cloud monopolies.
The most powerful AI shouldn't belong to a handful of corporations. Graviton is the open-source engine that breaks AI free from cloud monopolies and puts it on hardware you already own . One command. No cloud. No subscription. Just AI — for everyone. Models that corporations run on server farms — Graviton runs on your desk. A 72B model compressed from 144 GB to 36 GB, streamed layer by layer so it never exceeds your RAM. No OpenAI subscription. No API rate limits. No data harvesting. Your AI runs on your machine, owned by you, controlled by you. Graviton compresses AI models from 16-bit down to 4-bit or even 1.58-bit — making them 4 to 10 times smaller without losing the ability to think…from opengraviton.github.io
In the maker’s words, at launch
Hi HN, I built OpenGraviton, an open-source AI inference engine designed to push the limits of running extremely large models on consumer hardware. The system combines several techniques to drastically reduce memory and compute requirements: • 1.58-bit ternary quantization ({-1, 0, +1}) for ~10x compression • dynamic…
Does a similar job
all alternatives →
Run 500B+ Parameter LLMs Locally on a Mac MiniMar 2026 · github.com · ▲17Hi HN, I built OpenGraviton, an open-source AI inference engine that pushes the limits of running extremely large LLMs on consumer hardware. By combining 1.58-bit ternary quantization, dynamic sparsity with Top-K pruning and MoE routing, and mmap-based layer streaming, OpenGraviton can run models far larger than your system RAM—even on a Mac Mini. Early benchmarks: TinyLlama-1.1B drops from ~2GB (FP16) to ~0.24GB with ternary quantization. At 140B scale, models that normally require ~280GB fit within ~35GB packed. Optimized for Apple Silicon with Metal + C++ tensor unpacking, plus…
Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series MacJul 2026 · github.com · ▲919Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal. I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory. The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are…
General ComputeMay 2026 · generalcompute.com · ▲315AI models that run on an inference cloud optimized for speed
Moonshine Open-Weights STT models – higher accuracy than WhisperLargev3Feb 2026 · github.com · ▲316I wanted to share our new speech to text model, and the library to use them effectively. We're a small startup (six people, sub-$100k monthly GPU budget) so I'm proud of the work the team has done to create streaming STT models with lower word-error rates than OpenAI's largest Whisper model. Admittedly Large v3 is a couple of years old, but we're near the top the HF OpenASR leaderboard, even up against Nvidia's Parakeet family. Anyway, I'd love to get feedback on the models and software, and hear about what people might build with it.
Port of OpenAI's Whisper model in C/C++2022 · github.com · ▲399Hi HN, OpenAI recently released a model for automatic speech recognition called Whisper [0]. I decided to reimplement the inference of the model from scratch using C/C++. To achieve this I implemented a minimalistic tensor library in C and ported the high-level architecture of the model in C++. The entire code is less than 8000 lines of code and is contained in just 2 source files without any third-party dependencies. The Github project is here: https://github.com/ggerganov/whisper.cpp With this implementation I can very easily build and run the model - “make…
InvokeAI, an open source Stable Diffusion toolkit and WebUI2022 · github.com · ▲414Hey everyone! Excited to be able to share the release of `InvokeAI 2.0 - A Stable Diffusion Toolkit`, an open source project that aims to provide both enthusiasts and professionals a suite of robust image creation tools. Optimized for efficiency, InvokeAI needs only ~3.5GB of VRAM to generate a 512x768 image (and less for smaller images), and is compatible with Windows/Linux/Mac (M1 & M2). InvokeAI was one of the earliest forks off of the core CompVis repo (formerly lstein/stable-diffusion), and recently evolved into a full-fledged community driven and open source stable…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 30d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, March 2026
the whole month →
Switch from ChatGPT to Claude with import memory feature
AI · Mar 2026 · claude.com
Littlebird▲709Littlebird remembers everything across your meetings, messages, docs, and apps, then takes action so you can recall any detail and never lose context.
AI · Mar 2026 · littlebird.ai
