Beating Hinton et al.'s capsule net with fewer params and less training
Hello HN, I recently posted a work-in-progress paper, along with code necessary for replicating all its results, at: https://github.com/glassroom/heinsen_routing Among other things, the code in this repo outperforms Hinton et al.'s recent state-of-the-art result in visual recognition[0] while requiring fewer parameters and an order-of-magnitude fewer training epochs. Most of the original research we do at work tends to be either proprietary in nature or tightly coupled to internal code, so we cannot share it with the world. In this case, however, I was able to remove all…
What it does
In the maker’s words, at launch
Hello HN, I recently posted a work-in-progress paper, along with code necessary for replicating all its results, at: https://github.com/glassroom/heinsen_routing Among other things, the code in this repo outperforms Hinton et al.'s recent state-of-the-art result in visual recognition[0] while requiring fewer parameters and an order-of-magnitude fewer training epochs. Most of the original research we do at work tends to be either proprietary in nature or tightly coupled to internal code, so we cannot share it with the world. In this case, however, I was able to remove all traces of internal code and release this as stand-alone open-source software without having to disclose any key IP. I've reached out to academics in different groups for feedback, and the response so far has been positive, although most have only skimmed the paper. It will likely take a few weeks to get proper feedback from academia. In the meantime, I figured there are a lot of super-smart, knowledgeable people on HN who would love to take a look at this and share their thoughts. Please feel free to ask questions. Let me know what you think! [0] https://ai.google/research/pubs/pub46653
Does the same job
all alternatives →- L3Llama 3.2 Interpretability with Sparse Autoencoders2024 · github.com · ▲579
I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1]…
- HIHow I topped the HuggingFace open LLM leaderboard on two gaming GPUsMar 2026 · dnhkng.github.io · ▲495
I found that duplicating a specific block of 7 middle layers in Qwen2-72B, without modifying any weights, improved performance across all Open LLM Leaderboard benchmarks and took #1. As of 2026, the top 4 models on that leaderboard are still descendants. The weird finding: single-layer duplication does nothing. Too few layers, nothing. Too many, it gets worse. Only circuit-sized blocks of ~7 layers work. This suggests pretraining carves out discrete functional circuits in the layer stack that only work when preserved whole. The whole thing was developed on 2x RTX 4090s in my basement. I'm…
- ICI created a Neural Network from scratch, in scratch2024 · bell-boy.github.io · ▲7
I spent the past week implementing a 1 Layer Neural Net and training it on MNIST within the visual scripting language provided by scratch.mit.edu. It was tedious, but ultimately not too difficult. The code runs incredibly slowly, so much so that 64 samples of MNIST takes 5+ hours to train on my machine. There were a lot of little mini challenges that were fun to overcome (implementing softmax was very tricky). If you're interested, I encourage you to try and improve on it! More details in the linked blog post.
- ICI co-wrote a book on training Deep Learning models in Go2019 · ▲45
A friend and I wrote a book on how to build and train Deep Learning models in Go. We wanted it to be a useful reference for deep learning basics for Go programmers. Deep Learning is slowly seeping into everything we use every day and we thought it would be great if more people could do it in Go. The book is available here and on Amazon as well. https://www.packtpub.com/big-data-and-business-intelligence/hands-deep-learning-go We would appreciate any feedback and we're always looking to improve.
- IBI built a tool that helps predict HN front page successMay 2026 · wannalaunch.com · ▲26
Hey HN community, I built a tool that helps optimize your post for hitting the first page of Show HN. How it works: I used a Hugging Face dataset of all Hacker News posts from the past 3 years and trained a model that predicts how successful your post might be. There's still a lot of randomness on HN, so nothing is guaranteed, but the tool helps optimize your post for higher odds. A couple of interesting findings: - GitHub repo links work x3 better than regular domains - Open-source tools have a steady virality rate (13.9% - one of the highest) - "I built" outperforms "We built" - Using…
- W1Watch 14-Byte AI "brains" attempt to solve a 2D maze (Its hard)Jul 2026 · con-dog.github.io · ▲25
Hey HackerNews, I built this project over the last few weeks as a palette cleanser from a failed game launch. I wanted to learn a bit about AI/Neural-Networks and naively thought I could build a tiny maze-solving AI in a weekend with a 100% solve rate. Well - I couldn't, but I got pretty close. 14 Bytes total model size, and a 96.5% solve rate on unseen mazes. Trained across 46 phases experimenting with different ideas to improve the model (better performance, smaller size). Its quite fun to watch the model attempt to solve the maze, when they fail its usually due to getting stuck in a…
More dev tools this month
the category →
The first open-source price index for GPU compute Discussion | Link
Dev tools · 10d ago · producthunt.com
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 2d ago · opentrailpaper.com
Launched alongside, October 2019
the whole month →- BBel▲1,288
Life & fun · 2019 · paulgraham.com
Create free personalized illustrations from 3000+ elements
Growth · 2019 · icons8.com
