Linear RNN/Reservoir hybrid generative model, one C file (no deps.)
I just noticed it takes literally ~5 minutes to train millions parameters on slow CPU...but before you call Yudkowsky that "it's over", an important note: the main bottleneck is the corpus size, params are just 'cleverness' but given limited info it's powerless. Anyway, here is the project: https://github.com/bggb7781-collab/lrnnsmdds/tree/main couple of notes: 1. single C file, no dependencies. Below are literally all the "dependencies", not even custom header (copy paste from the top of the single c file): #define _POSIX_C_SOURCE 200809L #include #include…
What it does
In the maker’s words, at launch
I just noticed it takes literally ~5 minutes to train millions parameters on slow CPU...but before you call Yudkowsky that "it's over", an important note: the main bottleneck is the corpus size, params are just 'cleverness' but given limited info it's powerless. Anyway, here is the project: https://github.com/bggb7781-collab/lrnnsmdds/tree/main couple of notes: 1. single C file, no dependencies. Below are literally all the "dependencies", not even custom header (copy paste from the top of the single c file): #define _POSIX_C_SOURCE 200809L #include <stdio.h> #include <stdlib.h> #include <string.h> #include <math.h> #include <time.h> #include <stdint.h> #include <stdbool.h> #include <float.h> #include <getopt.h> #include <errno.h> 4136 lines of code in one file at the moment, that's all. 2. easiest way to compile on Windows: download Cygwin (https://www.cygwin.com/), then navigate to the directory where your lrnnsmdds.c file is and just run gcc on it with some optimizations, such as: gcc -std=c17 -O3 -march=native --fast-math -o lrnn lrnnsmdds.c -lm On Linux just run gcc, if for whatever reason you don't have gcc on Linux do sudo && apt-get install gcc --y ,or something... On Apple: i've no idea or maybe just use vmware and install ubuntu and then run it. Of course you can 'git clone' and go to the dir, but again: it's one file! copy it... The repo has tiny toy corpus included where i've borrowed (hopefully it's not plagiarism!) the name "John Gordon" from one of my favorite books "Star Kings", by E. Hamilton. Just the first and last name are copied, the content is unique (well several poorly written sentences by myself...). Obviously it will overfit and result on copy-paste on such small corpus, the sole goal is to check if everything runs and not if it's the A-G-I. You'd need your own 100kb+ if you want to generate unique meaningful text. 3. why/what/when/how? The github repo is self-explanatory i believe about features, uses and goals but in an attempt to summarize: My main motivation was to create a fast alternative to transformers which works on CPU only, hence you see the bizarre/not-easy task of doing this in C and not python and the lack of dependencies. In addition I was hoping it will also be clever alternative hence you see all those features more stacked than 90s BMW 850. The 'reservoir' is the most novel feature though, it offers quick exact recall arguably different than RWKV 8 or the latest Mamba, in fact name of the architecture SMDDS comes from the first letters of the implemented features: * S. SwiGLU in Channel Mixing (more coherence) * M. Multi-Scale Token Shift (larger context) * D. Data-Dependent Decay with Low-Rank (speed in large context) * D. Dynamic State Checkpointing (faster/linear generation) * S. Slot-memory reservoir (perfect recall, transformers style). If you face some issue just email me (easiest). the good, the bad the ugly: It is more or less working text-to-text novel alternative architecture, it's not trying to imitate transformers nor LSTM, Mamba, RWKV though it shares many features with them - the bad is that it's not blazing fast, if you're armed with ryzen/i7 16 cores or whatever and patience you can try training it on several small books via word tokenizer and low perplexity (under 1.2...) and see if it looks smarter/faster. Since this is open source obviously the hope is to be improved: make it cuda-friendly, improve the features, port to python etc. Depending on many factors I may try to push for v2 in July, August, September. My focus at the moment will be to test and scale since the features are many, it compiles with zero warnings on the 2 laptops i've tested(windows/cygwin and ubuntu) and the speed is comparable to transformers. 10x!
Does the same job
all alternatives →- IBI built a tiny LLM to demystify how language models workApr 2026 · github.com · ▲915
Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.
- III invented a new generative model and got accepted to ICLROct 2025 · discrete-distribution-networks.github.io · ▲656
I invented Discrete Distribution Networks, a novel generative model with simple principles and unique properties, and the paper has been accepted to ICLR2025! Modeling data distribution is challenging; DDN adopts a simple yet fundamentally different approach compared to mainstream generative models (Diffusion, GAN, VAE, autoregressive model): 1. The model generates multiple outputs simultaneously in a single forward pass, rather than just one output. 2. It uses these multiple outputs to approximate the target distribution of the training data. 3. These outputs together represent a discrete…
- SUSpeeding up LLM inference 2x times (possibly)2024 · asciinema.org · ▲419
Here's a project I've been working on for the last few months. It's a new (I think) algorithm, that allows to adjust smoothly - and in real time - how many calculations you'd like to do during inference of an LLM model. It seems that it's possible to do just 20-25% of weight multiplications instead of all of them, and still get good inference results. I implemented it to run on M1/M2/M3 GPU. The mmul approximation itself can be pushed to run 2x fast before the quality of output collapses. The inference speed is just a bit faster than Llama.cpp's, because the rest of implementation…
- IBI built a tensor library from scratch in C++/CUDA2025 · github.com · ▲119
Hi HN, Over the past few months, I've been building `dsc`, a tensor library from scratch in C++/CUDA. My main focus has been on getting the basics right, prioritizing a clean API, simplicity, and clear observability for running small LLMs locally. The key features are: - C++ core with CUDA support written from scratch. - A familiar, PyTorch-like Python API. - Runs real models: it's complete enough to load a model like Qwen from HuggingFace and run inference on both CUDA and CPU with a single line change[1]. - Simple, built-in observability for both Python and C++. Next on the roadmap is…
- N5NN-512 – Generate standalone C code for neural nets2020 · nn-512.com · ▲111
- ACAutograd.c – A tiny ML framework built from scratchDec 2025 · github.com · ▲85
built a tiny pytorch clone in c after going through prof. vijay janapa reddi's mlsys book: mlsysbook.ai/tinytorch/ perfect for learning how ml frameworks work under the hood :)
More dev tools this month
the category →
The first open-source price index for GPU compute
Dev tools · 10d ago · getcomputable.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 2d ago · opentrailpaper.com
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
Launched alongside, April 2026
the whole month →
- AG
Thought the resources for GPU arch were lacking, so here we are
Life & fun · Apr 2026 · jaso1024.com
- IB
Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.
AI · Apr 2026 · github.com
- BC
Life & fun · Apr 2026 · sam-burns.com
- IB
With social media and now AI, its important to keep the indie web alive. There are many people who write frequently. Blogosphere tries to highlight them by fetching the recent posts from personal blogs across many categories. There are two versions: Minimal (HN-inspired, fast, static): https://text.blogosphere.app/ Non-minimal: https://blogosphere.app/ If you don't find your blog (or your favorite ones), please add them. I will review and approve it.
AI · Apr 2026 · text.blogosphere.app