I built a deep learning engine from scratch in Python
I’ve spent the last few months building a deep learning engine completely from scratch in Python (using only math and random). What started as a basic linear algebra calculator project grew into a symbolic tensor system with autodiff, custom matrix ops, attention mechanisms, LayerNorm, GELU, and even a text generation demo trained on the Brown corpus. I'm still an undergrad, so my main goal is to deeply understand how deep learning actually works under the hood - gradients, attention, backpropagation, optimizers - by building it step-by-step with full visibility into everything, and without…
I built a deep learning engine from scratch in Python launched on April 16, 2025 with 30 votes, #730 of 7,818 launches that month and more than 90% of that year's launches. The site was still online in September 2026, 1 year on.
In plain words
This is a deep learning engine built from scratch in Python using only fundamental mathematics, created by an undergraduate to understand how neural networks work internally. It includes a symbolic tensor system, automatic differentiation, custom matrix operations, attention mechanisms, layer normalization, GELU activation, and a text generation demo. Designed for educational exploration rather than production use, it prioritizes transparency and step-by-step implementation of core concepts like backpropagation, gradients, and optimizers without relying on existing deep learning frameworks.
written from the facts on this page · September 2026
From the sources
An experimental transformer stack and symbolic computation engine built entirely from first principles in pure Python. - GitHub - whitegra/dolphin: An experimental transformer stack and symbolic computation engine built entirely from first principles in pure Python.
It implements deep learning architecture and training logic without relying on NumPy, PyTorch, or any external libraries. Every operation—tensor arithmetic, backpropagation, attention, and optimization—is executed through hand-written, minimal Python logic. The result is a transparent and flexible platform for understanding transformers, automatic differentiation, and the inner workings of modern ML, without hiding behind black-box libraries. Dolphin is a minimalist computing substrate for symbolic tensor operations and transformer-based neural networks. It is: Dolphin is built to help researchers, students, and curious developers explore how large language models and attention mechanisms…from github.com
In the maker’s words, at launch
I’ve spent the last few months building a deep learning engine completely from scratch in Python (using only math and random). What started as a basic linear algebra calculator project grew into a symbolic tensor system with autodiff, custom matrix ops, attention mechanisms, LayerNorm, GELU, and even a text…
Does a similar job
all alternatives →
I trained a deep learning model to build sites for me2019 · zecoda.com · ▲397
I wrote a minimal deep learning library from scratch in Python2021 · github.com · ▲25A minimalist deep learning library written from scratch in Python - sradc/SmallPebble
Deep Learning Framework from scratch (60 step tutorial)2022 · koki0702.github.io · ▲40Want to create a Deep Learning framework from scratch? Checkout this resource: https://koki0702.github.io/dezero-book/en/index.html In this book, you will create a deep learning framework called "DeZero" from scratch (from zero), which is the original framework of this book. With minimal code, the framework's modern features are realized. In this book, you will make this small - yet powerful enough - framework in a total of 60 steps. It will deepen your knowledge of modern frameworks such as PyTorch and TensorFlow.
Simple and comprehensive tutorials in Deep Learning using TensorFlow2018 · github.com · ▲157Simple and comprehensive tutorials in TensorFlow. Contribute to easy-tensorflow/easy-tensorflow development by creating an account on GitHub.
More dev tools this month
the category →
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 27d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 25d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 22d ago · openbot.run
Talanoa▲340Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 20d ago · talanoa.email
CoreSpeed▲332One MCP for everything your agents need: apps, memory, tools
Dev tools · 27d ago · corespeed.io
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, April 2025
the whole month →
Hi everyone, I built PyXL — a hardware processor that executes a custom assembly generated from Python programs, without using a traditional interpreter or virtual machine. It compiles Python -> CPython Bytecode -> Instruction set designed for direct hardware execution. I’m sharing an early benchmark: a GPIO test where PyXL achieves a 480ns round-trip toggle — compared to 14-25 micro seconds on a MicroPython Pyboard - even though PyXL runs at a lower clock (100MHz vs. 168MHz). The design is stack-based, fully pipelined, and preserves Python's dynamic typing without static type restrictions.…
Dev tools · 2025 · runpyxl.com
The Unsure Calculator is an online tool that lets you calculate with numbers you’re not sure about.
Life & fun · 2025 · filiph.github.io
https://the-pocket.github.io/Tutorial-Codebase-Knowledge/
AI · 2025 · github.com
