I built a lite LPU that can do inference on Karpathy's MicroGPT
We had no guide or course that teaches chip design at our university. We had taken a digital logic course, but were disappointed with the fact that the most complex project we did was building a full adder in Quartus using logic blocks, not even in RTL!5 Therefore, we decided to challenge ourselves to dive deep into machine learning (ML) hardware and learn as much as we could on our own. We wanted to prove that basic math (like y = mx + b) and basic logic circuits are enough to help anyone understand how modern AI hardware works. Our goal was to design our own version of the LPU from scratch…
In plain words
LPU Lite is a language processing unit designed by students to run inference on small transformer models like MicroGPT. Built from scratch without prior chip design experience, it demonstrates that fundamental mathematics and logic circuits are sufficient to understand AI hardware architecture. The project aims to demystify specialized processor design by recreating a simplified version of Groq's proprietary LPU, proving that basic knowledge of digital logic can enable anyone to build functional machine learning hardware.
written from the facts on this page · September 2026
From the sources
LPU Lite: a language processing unit built from first principles.
a lite language processing unit by Saksham Batra, Michael Trbovic, and Arjun Harinath. With no previous chip design knowledge, we decided to figure it out ourselves and see how far we could get! We wanted to demystify AI hardware by reinventing a fully functional LPU from scratch to run a Transformer! LPU stands for Language Processing Unit. It is a specialized computer chip developed by Groq 1 specifically for running large language models such as ChatGPT and Claude. Groq has kept the secret of their LPU closed-source and only left small hints in their research paper. 2 , 3 The LPU differs from a traditional chip in many ways, but the overarching idea is “deterministic execution.”…from lpulite.com
In the maker’s words, at launch
We had no guide or course that teaches chip design at our university. We had taken a digital logic course, but were disappointed with the fact that the most complex project we did was building a full adder in Quartus using logic blocks, not even in RTL!5 Therefore, we decided to challenge ourselves to dive deep into machine learning (ML) hardware and learn as much as we could on our own. We wanted to prove that basic math (like y = mx + b) and basic logic circuits are enough to help anyone understand how modern AI hardware works. Our goal was to design our own version of the LPU from scratch and run a simple Transformer-style model on it, proving that with minimal Machine Learning and computer design knowledge, it’s totally possible. We were also driven by a simple question: What makes the LPU architecture so compelling that even Nvidia licensed it? Keep in mind, this article is not intended to serve as a tutorial for “how to build an LPU from scratch,” and our architecture is not a 1:1 LPU. It serves as an educational resource for how someone with minimal hardware experience can approach this field, and our journey in building what we think an LPU would look like.
Does the same job
all alternatives →
- IBI built a tiny LLM to demystify how language models workApr 2026 · github.com · ▲915
Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.
- IBI built a toy TPU that can do inference and training on the XOR problem2025 · tinytpu.com · ▲134
We wanted to do something very challenging to prove to ourselves that we can do anything we put our mind to. The reasoning for why we chose to build a toy TPU specifically is fairly simple: - Building a chip for ML workloads seemed cool - There was no well-documented open source repo for an ML accelerator that performed both inference and training None of us have real professional experience in hardware design, which, in a way, made the TPU even more appealing since we weren't able to estimate exactly how difficult it would be. As we worked on the initial stages of this project, we…
- ATA tiny LLM running at 21,000 tok/s on a $250 FPGA (Live Demo)27d ago · mikeayles.com · ▲79
A 3.16M-parameter INT4 transformer running entirely in the on-chip memory of a Xilinx Kria KV260. Zero DRAM in the token loop, 59,965 tok/s on the fabric, bit-exact. Chat with it live.
- TVTiny-vLLM – high performance LLM inference engine in C++ and CUDAMay 2026 · github.com · ▲205
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, August 2026
the whole month →- TL
Life & fun · 10d ago · louisabraham.github.io
- SA
Hello HN! I found that picking out plausible but diverse skin tones for my digital art and game development projects was kind of difficult, and I got curious about if there was a way to define a color space that made it easy. I've built a color picker and procedural generation algorithm based on the space as well as a bunch of other fun js features and demos throughout the page that use the equations. If you find it interesting, I have lots of explanations of how I built it and what properties the space has. The methodology might be a bit shaky, but hopefully the result is as helpful for…
Life & fun · Aug 2026 · toneyalexander.github.io
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com