Learning to (Learn at Test Time)
I’m excited to share a project I’ve been working on for over a year, which I believe will fundamentally change our approach to language models. We’ve designed a new architecture, which replaces the hidden state of an RNN with a machine learning model. This model compresses context through actual gradient descent on input tokens. We call our method “Test-Time-Training layers.” TTT layers directly replace attention, and unlock linear complexity architectures with expressive memory, allowing us to train LLMs with millions (someday billions) of tokens in context. Our instantiations, TTT-Linear…
Learning to (Learn at Test Time) launched on July 8, 2024 with 13 votes, #1,242 of 5,184 launches that month and more than 75% of that year's launches.
In plain words
Learning to (Learn at Test Time) is a language model architecture that replaces traditional attention mechanisms with Test-Time-Training layers, which compress context through gradient descent on input tokens. Designed for researchers and developers building large language models, it enables linear complexity while maintaining expressive memory, allowing training with millions of tokens in context. The method achieves performance comparable to or better than Transformers and Mamba while offering improved efficiency for handling extended sequences.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
I’m excited to share a project I’ve been working on for over a year, which I believe will fundamentally change our approach to language models. We’ve designed a new architecture, which replaces the hidden state of an RNN with a machine learning model. This model compresses context through actual gradient descent on…
Does a similar job
all alternatives →
I built a tiny LLM to demystify how language models workApr 2026 · github.com · ▲915Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.
Gan.AI TTS Model & API Playground2024 · ▲333First TTS model to support all 22 Indic languages + English
Duplicate 3 layers in a 24B LLM, logical deduction .22→.76. No trainingMar 2026 · github.com · ▲265I replicated David Ng's RYS method (https://dnhkng.github.io/posts/rys/) on consumer AMD GPUs (RX 7900 XT + RX 6950 XT) and found something I didn't expect. Transformers appear to have discrete "reasoning circuits" — contiguous blocks of 3-4 layers that act as indivisible cognitive units. Duplicate the right block and the model runs its reasoning pipeline twice. No weights change. No training. The model just thinks longer. The results on standard benchmarks (lm-evaluation-harness, n=50): Devstral-24B, layers 12-14 duplicated once: - BBH Logical Deduction: 0.22 → 0.76…
Inworld TTS2025 · inworld.ai · ▲318Hey HN, this is Igor, one of the engineers behind Inworld Runtime, which we're releasing today in public preview. We built it to solve the common problem we and our customers had: engineers spend more time on AI ops and plumbing than on actual feature development. This was often due to the challenge of using Python for I/O-bound, high-concurrency workloads and complexity maintaining pipelines with streams that use always-changing ML models. Our solution is a high-performance runtime written in C++ with the core idea of defining AI logic as graphs. For instance, a basic voice-to-voice…
A perceptron with stochastic gradient descent in Go2017 · github.com · ▲94A single / multi layer / recurrent neural network written in Golang. - made2591/go-perceptron-go
- IBI built a toy TPU that can do inference and training on the XOR problem2025 · tinytpu.com · ▲134
We wanted to do something very challenging to prove to ourselves that we can do anything we put our mind to. The reasoning for why we chose to build a toy TPU specifically is fairly simple: - Building a chip for ML workloads seemed cool - There was no well-documented open source repo for an ML accelerator that performed both inference and training None of us have real professional experience in hardware design, which, in a way, made the TPU even more appealing since we weren't able to estimate exactly how difficult it would be. As we worked on the initial stages of this project, we…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 21d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, July 2024
the whole month →
Many years ago, I made VJ softwares (to mix live visuals in clubs) for unexpected platforms like the Game Boy Advance, the Playstation 2 and the Raspberry Pi. This year, I’m back with a new web-app: Pikimov. Inspired by Photopea (a free Photoshop clone), I created this web-based motion design & video editor as an alternative to After Effects, to fill empty void. It's free, without signup, without cloud uploads (your files stay on your machine), and your projects are not used for AI models training.
AI · 2024 · pikimov.com
