Alternatives
Products that do what I wrote an autodiff in C++ and implemented LeNet with it does
- 1BG
Trying to do gradient descent using automatic differentiation over branchy programs? Or to combine them with neural networks for end-to-end training? Then this might be interesting to you. We develped DiscoGrad, a tool for automatic differentiation through C++ programs involving input-dependent control flow (e.g., "if (f(x) < c) { ... }", differentiating wrt. x) and randomness. Our initial motivation was to enable the use of gradient descent with simulations, which often rely heavily on such discrete branching. The latter makes plain autodiff mostly useless, since it can only account for the…
2024 · github.com
- 2L3
I spent a lot of time and money on this rather big side project of mine that attempts to replicate the mechanistic interpretability research on proprietary LLMs that was quite popular this year and produced great research papers by Anthropic [1], OpenAI [2] and Deepmind [3]. I am quite proud of this project and since I consider myself the target audience for HackerNews did I think that maybe some of you would appreciate this open research replication as well. Happy to answer any questions or face any feedback. Cheers [1]…
2024 · github.com
- 3MM
Jun 2026 · github.com
- 4AH
Long-simmering side project that is finally ready to see the light. HAMTs are a cool persistent data structure and implementing one has been a lot of fun. Beyond the code, there is likely some value in the extensive and largely complete implementation docs; basic benchmarks are linked in the README, too. Kind of aiming to be "the libavl for HAMTs". That is obviously a high and aspirational bar but a distinct possibility if it stirs up a little interest and/or contribution. Anyways, it's time for this to go out, collect feedback and maybe even some use outside of toy projects. Let me…
2023 · github.com
- 5AC
built a tiny pytorch clone in c after going through prof. vijay janapa reddi's mlsys book: mlsysbook.ai/tinytorch/ perfect for learning how ml frameworks work under the hood :)
Dec 2025 · github.com
- 6AD
2019 · github.com
- 7AA
2016 · github.com
- 8VA
I was fascinated reading through another recent HN submission about a highly efficient implementation of A* in Lisp, which got me thinking about how I could do something similar in Python. However, these kinds of pathfinding algorithms really need complex terrain/mazes with interesting obstructions to showcase what they can do and how they work. So, I started thinking about how I could generate cool and diverse random "mazes" (they aren't really mazes, but I'm not sure what the best term is). I got a bit carried away thinking of lots of different cool ways to generate these mazes, such…
2024 · github.com
- 9TB
2017 · github.com
- 10IB
I’ve spent the last few months building a deep learning engine completely from scratch in Python (using only math and random). What started as a basic linear algebra calculator project grew into a symbolic tensor system with autodiff, custom matrix ops, attention mechanisms, LayerNorm, GELU, and even a text generation demo trained on the Brown corpus. I'm still an undergrad, so my main goal is to deeply understand how deep learning actually works under the hood - gradients, attention, backpropagation, optimizers - by building it step-by-step with full visibility into everything, and without…
2025 · github.com
- 11LA
2014 · github.com
- 12AM
Hi all, I’m hacking on new features for the ClickHouse native client and wanted the same “just call the model” ergonomics JavaScript and Python now enjoy. It didn’t exist for modern C++, so I wrote one. ai‑sdk‑cpp (Apache‑2.0) gives you: - Unified calls to OpenAI (GPT‑4o) and Anthropic (Claude 3.5) with a single C++20 API. - Streaming, multi‑turn chat, error handling—all std::optional/std::variant, no macros. - Tool calling (function‑calling) so the model can hit real APIs; sync or async, runs in parallel. The tricky bit: C++ still lacks real reflection, so mapping plain functions →…
2025
- 13MD
Hello everyone! I've been working on this project for a few months as part of my thesis in Machine Learning. It's meant to be a library that provides an easy-to-use but flexible API to design and train Diffusion Models. I decided to make it because I wanted to quickly prototype a Diffusion Model but there were no good tools to do it with. I think it really can help people prototype their own Diffusion Models a lot faster and only in a few lines of code. The base idea is to have a Model class that takes different modules corresponding to the different aspects of the Diffusion Model process…
2023 · github.com
- 14ZA
This is a low-level opensource library I developed for my own use and decided to share, as it makes it possible to process large checkpoints of neural networks without renting high-RAM instances, on a regular PC. It replaces torch.load() with a custom function that produces a dictionary that materializes tensors on the fly. Compared to other solutions it doesn't require sharding or re-encoding checkpoints and uses them completely as-is. It is a foundation to make it possible to run inference and compress language models and other large models one layer at a time - in principle, even one…
2023 · github.com
- 15IW
2021 · github.com
- 16SG
Hi HN, I built NOMA (Neural-Oriented Machine Architecture), a systems language where reverse-mode autodiff is a compiler pass (lowered to LLVM IR). My goal is to treat model parameters as explicit, growable memory buffers. Since NOMA compiles to standalone native binaries (no Python runtime), it allows using realloc on weights mid-training. This makes "self-growing" architectures a system primitive rather than a complex framework hack. I just pushed a reproducible benchmark (Self-Growing XOR) to validate the methodology: it compares NOMA against PyTorch and C++, specifically testing how…
Dec 2025 · github.com
- 17IC
I spent the past week implementing a 1 Layer Neural Net and training it on MNIST within the visual scripting language provided by scratch.mit.edu. It was tedious, but ultimately not too difficult. The code runs incredibly slowly, so much so that 64 samples of MNIST takes 5+ hours to train on my machine. There were a lot of little mini challenges that were fun to overcome (implementing softmax was very tricky). If you're interested, I encourage you to try and improve on it! More details in the linked blog post.
2024 · bell-boy.github.io
- 18FS
I want to share a really dumb, but very practical project I have packaged this summer, to perform operations on strings much faster. I was using Python to work with a multi-terabyte newline-delimited file. Reading, splitting, and shuffling it was a nightmare. So, I wrapped a trivial hardware-friendly heuristic I've been using for the last few years into a CPython library. The part I enjoyed the most is implementing SIMD behavior without SIMD instructions... Using 64-bit words to work at 8-bit granularity. Unlike conventional SIMD, the code would remain the same for ~~almost~~ any hardware.…
2023 · ashvardanian.com
- 19MD
Dec 2025 · github.com
- 20MI
The release of microgpt by Andrej Karpathy is a foundational moment for AI transparency. In exactly 243 lines of pure, dependency-free Python, Karpathy has implemented the complete GPT algorithm from scratch. As a PhD scholar investigating AI and Blockchain, I see this as the ultimate tool for moving beyond the "black box" narrative of Large Language Models (LLMs). The Architecture of Simplicity Unlike modern frameworks that hide complexity behind optimized CUDA kernels, microgpt exposes the raw mathematical machinery. The code implements: The Autograd Engine: A custom Value class that…
Feb 2026
- 21IM
I’m 15 and self-taught. I'm learning ML from scratch because I want to really understand how things work. I’m not into frameworks. I prefer math, logic, and C++. I implemented a basic MLP that supports different activation and loss functions. It was trained via mini-batch gradient descent. I wrote it from scratch, using no external libraries except Eigen (for linear algebra). I learned how a Neural Network learns (all the math) -- how the forward pass works, and how learning via backpropagation works. How to convert all that math into code. I’ll write a blog soon explaining how MLPs work in…
2025 · github.com
- 22AT
May 2026 · github.com
- 23LT
I’m excited to share a project I’ve been working on for over a year, which I believe will fundamentally change our approach to language models. We’ve designed a new architecture, which replaces the hidden state of an RNN with a machine learning model. This model compresses context through actual gradient descent on input tokens. We call our method “Test-Time-Training layers.” TTT layers directly replace attention, and unlock linear complexity architectures with expressive memory, allowing us to train LLMs with millions (someday billions) of tokens in context. Our instantiations, TTT-Linear…
2024
- 24DA
Lately a friend of mine has been working on a personal project and I wanted to share it. The goal of Diffulab is to provide a flexible and modular framework for training diffusion models from scratch. The project is still in its early stages, and he is actively working on adding new features and improvements.
2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →