Life & fun · alternatives · 2026
24 alternatives to Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone
Swiftlet is a Swift and Metal runtime that runs large Qwen Mixture-of-Experts models locally on Apple devices by streaming expert weights from storage, enabling 35B and 80B models to run with low RAM, including on iPhone. - leonickson1/Swiftlet
This tool enables users to run large language models locally on resource-constrained devices. It allows an 80-billion-parameter Qwen model to operate on a Mac with only 4.3 GB of RAM, and a 35-billion-parameter version… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift. It ships with auto-mode, which makes a good tradeoff between memory usage and speed. I'll be implementing and porting the MTP module for speculative decoding next Local models really are the future of computing!
Sep 2026 · github.com · its alternatives →
- 2
Hi HN, I built a specialized inference engine for running 4-bit Gemma 4 26B-A4B-IT on any M-series Mac using about 2 GB of RAM. It is called TurboFieldfare and is written in Swift and Metal. I have always adored on-device AI. It feels like magic that you can run a powerful NN on your Mac or iPhone. So I wanted to push the limits a bit and run a model whose weights don’t fit in memory. The model’s 4-bit quantized weights occupy roughly 14 GB, which makes running it with conventional inference tools almost impossible on an 8 GB or even 16 GB Mac once the OS, applications, and KV cache are…
Jul 2026 · github.com · its alternatives →
- 3
Qwen3.5 Small▲3550.8B-9B native multimodal w/ more intelligence, less compute
Mar 2026 · huggingface.co · its alternatives →
- 4
Qwen3.8-Max▲277Qwen3.8-Max + MCP for coding on your local machine, without paying for Qwen Code. Qwen3.8-Max itself runs in the cloud through Qwen Studio — this setup just gives it access to your local files and terminal through MCP. My take: it is slower than Codex and Claude Code, especially in thinking mode, but it does work well. MCP adds some latency too since there’s an extra layer between the model and your machine. In fast mode, it gets noticeably closer. I wouldn’t put it in the same category as Fable or Opus-5. Personally, I’d place it closer to the next Opus 4.7 level in terms of coding ability.…
Aug 2026 · qwen.ai · its alternatives →
- 5
We built SwiftAI, an open-source Swift library that lets you use Apple’s on-device LLMs when available (Apple opened access in June), and fall back to a cloud model when they aren’t available — all without duplicating code. SwiftAI gives you: - A single, model-agnostic API - An agent/tool loop - Strongly-typed structured outputs - Optional chat state Backstory: We started experimenting with Apple’s local models because they’re free (no API calls), private, and work offline. The problem: not all devices support them (older iPhones, Apple Intelligence disabled, low battery, etc.). That…
2025 · github.com · its alternatives →
- 6
Contribute to Mega4alik/ollm development by creating an account on GitHub.
2025 · github.com · its alternatives →
- 7
Run Qwen's latest models locally on your iPhone
Mar 2026 · locallyai.app · its alternatives →
- 8MP
Aug 2026 · deepgrove.ai · its alternatives →
- 9
Qwen3.5▲288The 397B native multimodal agent with 17B active params
Feb 2026 · qwen.ai · its alternatives →
- 10
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Sep 2026 · huggingface.co · its alternatives →
- 11
Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…
2023 · github.com · its alternatives →
- 12
- 13
Osaurus▲534Open source agents that run 100% locally on your Mac
Jul 2026 · osaurus.ai · its alternatives →
- 14
I ported Microsoft's TRELLIS.2 (4B parameter image-to-3D model) to run on Apple Silicon via PyTorch MPS. The original requires CUDA with flash_attn, nvdiffrast, and custom sparse convolution kernels: none of which work on Mac. I replaced the CUDA-specific ops with pure-PyTorch alternatives: a gather-scatter sparse 3D convolution, SDPA attention for sparse transformers, and a Python-based mesh extraction replacing CUDA hashmap operations. Total changes are a few hundred lines across 9 files. Generates ~400K vertex meshes from single photos in about 3.5 minutes on M4 Pro (24GB). Not as fast as…
Apr 2026 · github.com · its alternatives →
- 15
Metal-first MoE inference for Apple Silicon with bounded SSD expert streaming and local OpenAI Chat, Responses, and Anthropic Messages endpoints. - hebrus-labs/hebrus
Jul 2026 · github.com · its alternatives →
- 16
Last week was big for open source LLMs. We got: - Qwen 2.5 VL (72b and 32b) - Gemma-3 (27b) - DeepSeek-v3-0324 And a couple weeks ago we got the new mistral-ocr model. We updated our OCR benchmark to include the new models. We evaluated 1,000 documents for JSON extraction accuracy. Major takeaways: - Qwen 2.5 VL (72b and 32b) are by far the most impressive. Both landed right around 75% accuracy (equivalent to GPT-4o’s performance). Qwen 72b was only 0.4% above 32b. Within the margin of error. - Both Qwen models passed mistral-ocr (72.2%), which is specifically trained for OCR. - Gemma-3…
2025 · github.com · its alternatives →
- 17
- 18
Qwen3-Coder▲248A powerful open model for agentic coding tasks
2025 · qwenlm.github.io · its alternatives →
- 19
- 20
High-speed download of LLaMA, Facebook's 65B parameter GPT model - shawwn/llama-dl
2023 · github.com · its alternatives →
- 21
Qwen's most advanced reasoning model yet
2025 · huggingface.co · its alternatives →
- 22
- 23
First TTS model to support all 22 Indic languages + English
2024 · its alternatives →
- 24TF
I’d originally launched my app: Private LLM[1][2] on HN around 10 months ago, with a single RedPajama Chat 3B model. The app has come a long way since then. About a month ago, I added support for 4-bit OmniQuant quantized Mixtral 8x7B Instruct model, and it seems to outperform Q4 models at inference speed and Q8 models at text generation quality, while consuming only about 24GB of RAM[3] at 8k context length. The trick is: a) to use a better quantization algorithm and b) to use unquantized embeddings and the MoE gates (the overhead is quite small). Other notable features include many more…
2024 · its alternatives →
Also compare
- Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/s alternatives
- Open-source engine running Gemma 4 26B in 2 GB RAM on any M-series Mac alternatives
- Qwen3.5 Small alternatives
- Qwen3.8-Max alternatives
- SwiftAI – open-source library to easily build LLM features on iOS/macOS alternatives
- Run Qwen3-Next-80B on 8GB GPU at 1tok/2s throughput alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →