GPT Circuits – Mapping the inner workings of simple LLMs
I've built an app that extracts interpretable 'circuits' from models using the GPT-2 architecture. These circuits reveal how specific inputs influence the probabilities of the next token in a sequence. While some tutorials present theoretical examples of how feedforward layers and attention heads may produce predictions, this app provides concrete examples of how information flows through an LLM. You can see, for example, the formation of features that search for simple grammatical patterns and trace their construction back to the use of more primitive features. Feel free to reach out with…
What it does
In the maker’s words, at launch
I've built an app that extracts interpretable 'circuits' from models using the GPT-2 architecture. These circuits reveal how specific inputs influence the probabilities of the next token in a sequence. While some tutorials present theoretical examples of how feedforward layers and attention heads may produce predictions, this app provides concrete examples of how information flows through an LLM. You can see, for example, the formation of features that search for simple grammatical patterns and trace their construction back to the use of more primitive features. Feel free to reach out with feedback. I'd love to work with others on scaling out the size of the model that circuits can be extracted from.
Does the same job
all alternatives →
- ATA tiny reasoning layer that steadies LLM outputs (MIT; +22.4% accuracy)2025 · github.com · ▲6
We kept shipping “simple” LLM features that were fluent-but-wrong. After too many postmortems we wrote down the failure patterns and added a small reasoning layer in front of the model. It’s model-agnostic, sits beside your existing stack, and you can implement it from a single PDF (MIT). What’s inside the PDF A problem map of 16 failure modes we kept hitting in real systems (OCR/layout drift, table-to-question mismatches, embedding≠meaning, pre-deploy collapse, etc.). Four lightweight gates you can add today: Knowledge-boundary canaries (empty/adversarial/known-fact probes).…
- KAKnowing – an LLM tool built on concept hierarchies, not prompt-response2024 · ▲5
Hey HN! I've spent the past year full-time building Knowing, a tool for interacting with LLMs directly inside hierarchical structures instead of the usual prompt-response format. The idea started because I realized how much more intuitive it felt to build concept hierarchies continuously—no more endless copy-pasting or wondering how everything connects. The journey’s been a struggle. While I see huge potential in structuring AI interactions this way (writing books fast, planning projects, or organizing ideas), it’s been hard to pin down clear use cases in the market. I’m also working in near…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, October 2024
the whole month →
Softr for Notion▲925Turn Notion databases into portals & apps with no code
Dev tools · 2024 · softr.io
One inbox for all your work discussions
Work · 2024 · generalcollaboration.com