I built a deep learning engine from scratch in Python
I’ve spent the last few months building a deep learning engine completely from scratch in Python (using only math and random). What started as a basic linear algebra calculator project grew into a symbolic tensor system with autodiff, custom matrix ops, attention mechanisms, LayerNorm, GELU, and even a text generation demo trained on the Brown corpus. I'm still an undergrad, so my main goal is to deeply understand how deep learning actually works under the hood - gradients, attention, backpropagation, optimizers - by building it step-by-step with full visibility into everything, and without…
In plain words
This is a deep learning engine built from scratch in Python using only fundamental mathematics, created by an undergraduate to understand how neural networks work internally. It includes a symbolic tensor system, automatic differentiation, custom matrix operations, attention mechanisms, layer normalization, GELU activation, and a text generation demo. Designed for educational exploration rather than production use, it prioritizes transparency and step-by-step implementation of core concepts like backpropagation, gradients, and optimizers without relying on existing deep learning frameworks.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
I’ve spent the last few months building a deep learning engine completely from scratch in Python (using only math and random). What started as a basic linear algebra calculator project grew into a symbolic tensor system with autodiff, custom matrix ops, attention mechanisms, LayerNorm, GELU, and even a text generation demo trained on the Brown corpus. I'm still an undergrad, so my main goal is to deeply understand how deep learning actually works under the hood - gradients, attention, backpropagation, optimizers - by building it step-by-step with full visibility into everything, and without relying on big frameworks or libraries. It’s not fast or production-ready, but that’s not the point. As of now, it’s more so aimed at exploration and understanding. I mainly wanted to explore how deep learning works by building it through first principles. It’s still a work in progress (lots to learn and improve in terms of structure, docs, and performance), but I figured it was worth sharing. I’d love any feedback, questions, ideas, or even just thoughts about what you’d add, change, or do differently. Thanks for reading!
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, April 2025
the whole month →- IB
Hi everyone, I built PyXL — a hardware processor that executes a custom assembly generated from Python programs, without using a traditional interpreter or virtual machine. It compiles Python -> CPython Bytecode -> Instruction set designed for direct hardware execution. I’m sharing an early benchmark: a GPIO test where PyXL achieves a 480ns round-trip toggle — compared to 14-25 micro seconds on a MicroPython Pyboard - even though PyXL runs at a lower clock (100MHz vs. 168MHz). The design is stack-based, fully pipelined, and preserves Python's dynamic typing without static type restrictions.…
Dev tools · 2025 · runpyxl.com
- UC
Life & fun · 2025 · filiph.github.io
- IB
https://the-pocket.github.io/Tutorial-Codebase-Knowledge/
AI · 2025 · github.com
