OS Megakernel that match M5 Max Tok/w at 2x the Throughput on RTX 3090

What it does

LLM speculative inference server for heterogeneous hardware & consumer GPUs - Luce-Org/lucebox

In the maker’s words, at launch

Hey there, we fused all 24 layers of Qwen3.5-0.8B (a hybrid DeltaNet + Attention model) into a single CUDA kernel launch and made it open-source for everyone to try it. On an RTX 3090 power-limited to 220W: - 411 tok/s vs 229 tok/s on M5 Max (1.8x) - 1.87 tok/J, beating M5 Max efficiency - 1.55x faster…

• 
  Llama 3.1 70B on a single RTX 3090 via NVMe-to-GPU bypassing the CPUFeb 2026 · github.com · ▲395

  Hi everyone, I'm kinda involved in some retrogaming and with some experiments I ran into the following question: "It would be possible to run transformer models bypassing the cpu/ram, connecting the gpu to the nvme?" This is the result of that question itself and some weekend vibecoding (it has the linked library repository in the readme as well), it seems to work, even on consumer gpus, it should work better on professional ones tho

• 
  80% faster, 50% less memory, 0% loss of accuracy Llama finetuning2023 · github.com · ▲385

  Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original implementation on your own local GPU. Some highlights: 1. Manual autograd engine - hand derived backprop steps. 2. QLoRA / LoRA 80% faster, 50% less memory. 3. All kernels written in OpenAI's Triton language. 4. 0% loss in accuracy - no approximation methods - all exact. 5. No change of hardware necessary. Supports NVIDIA GPUs since 2018+. CUDA 7.5+. 6. Flash Attention support via Xformers. 7. Supports 4bit and 16bit…

• 
  Run Qwen3-Next-80B on 8GB GPU at 1tok/2s throughput2025 · github.com · ▲123

  Contribute to Mega4alik/ollm development by creating an account on GitHub.

• 
  Running 104GB Qwen3.8-Flash-Next on 48GB Mac with at ~12 tok/sSep 2026 · github.com · ▲240

  I built slotstream, a way to run Qwen3.8-Flash-Next 4-bit on a low-memory mac starting from 16GB, a 125B parameter model that would need 100GB+ memory/RAM, thanks to expert-offloading/ssd-streaming. Easy to install/update, and mac-native using MLX and Swift. It ships with auto-mode, which makes a good tradeoff between memory usage and speed. I'll be implementing and porting the MTP module for speculative decoding next Local models really are the future of computing!

• 
  Computable GPU Index (CGI)Aug 2026 · getcomputable.com · ▲444

  Hey we are Computable. We spent years building trading infrastructure at Jump Trading and Coinbase. From that point of view, compute looks like energy markets before 2000: everything trades through private bilateral leases, there’s no visible price, and nothing can be resold. The same H100 rents at a 2x spread depending on who’s asking, and once you sign a 24-month lease, it can never change hands. So we built a market for GPU nodes, sold by the calendar week. Here are three things you can do on it that you can’t do anywhere else: - Buy exactly the weeks you need. Three nodes for the last…

• 
  Soup CLIAug 2026 · trysoup.dev · ▲107

  Fine-tune an 8B LLM on a 4 GB laptop GPU

• 
  Run full iOS 16.5 firmware on Android Devices▲519

  Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.

  Dev tools · 2d ago · github.com

• 
  /dev/null▲492

  a social network for devs

  Dev tools · 5d ago · dev-null.sh

• 
  Whiteboard (YC W26) – an open-source IDE for thoughtful software design▲423

  open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.

  Dev tools · 30d ago · github.com

• 
  Reladraw – A diagram language where you decide where to place things▲411

  Contribute to reladraw/reladraw development by creating an account on GitHub.

  Dev tools · 28d ago · github.com

• 
  OpenBot▲362

  Grok Bot alternative: free, local, open-source, multiplayer

  Dev tools · 25d ago · openbot.run

• 
  Talanoa▲346

  Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…

  Dev tools · 23d ago · talanoa.email

• 
  Brila▲1,364

  One-page websites from real Google Maps reviews

  AI · Apr 2026 · brila.ai

• AG
  A game where you build a GPU▲964

  Thought the resources for GPU arch were lacking, so here we are

  Life & fun · Apr 2026 · jaso1024.com

• 
  I built a tiny LLM to demystify how language models work▲915

  Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.

  AI · Apr 2026 · github.com

• 
  Fathom 3.0▲800

  AI meeting notes: now bot-free, in ChatGPT & Claude + more

  AI · Apr 2026 · fathom.ai

• 
  Brutalist Concrete Laptop Stand (2024)▲786

  I made a laptop stand out of solid concrete, in the style of brutalist architecture, with themes of urbex and urban decay

  Life & fun · Apr 2026 · sam-burns.com

• 
  I built a frontpage for personal blogs▲783

  With social media and now AI, its important to keep the indie web alive. There are many people who write frequently. Blogosphere tries to highlight them by fetching the recent posts from personal blogs across many categories. There are two versions: Minimal (HN-inspired, fast, static): https://text.blogosphere.app/ Non-minimal: https://blogosphere.app/ If you don't find your blog (or your favorite ones), please add them. I will review and approve it.

  AI · Apr 2026 · text.blogosphere.app