Llama 3.2 Interpretability with Sparse Autoencoders
I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1]…
In plain words
This project replicates mechanistic interpretability research on large language models using sparse autoencoders, building on published work from Anthropic, OpenAI, and DeepMind. It applies techniques for understanding how neural networks process information to the open-source Llama 3.2 model. The project is aimed at researchers and developers interested in AI interpretability, offering an open-source alternative to proprietary LLM analysis work.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1] https://transformer-circuits.pub/2024/scaling-monosemanticit... [2] https://arxiv.org/abs/2406.04093 [3] https://arxiv.org/abs/2408.05147
More growth this month
the category →
AstraPixels▲267A pixel-art solar system at its real current positions.
Growth · 29d ago · astrapixels.com
Launched alongside, November 2024
the whole month →
- IB
I wasn't quite sure if this qualified as "Show HN" given you can't really download it and try it out. However, dang said[0]: > If it's hardware or something that's not so easy to try out over the internet, find a different way to show how it actually works—a video, for example, or a detailed post with photos. Hopefully I did that? Additionally, I've put code and a detailed guide for the netboot computer management setup on GitHub: https://github.com/kentonv/lanparty Anyway, if this shouldn't have been Show HN, I apologize! [0]…
Dev tools · 2024 · lanparty.house
