AI · alternatives · 2026

24 alternatives to NanoRL – RL training for LLMs in ~1,800 lines

The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster (e.g. 8xH100 trainer, 8 vLLM workers, ran as a [SkyPilot job group](https://docs.skypilot.ai/en/latest/examples/job-groups.html) on k8s ). All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.

NanoRL is a lightweight reinforcement learning trainer for large language models written in approximately 1,800 lines of code. It runs the same training loop on both laptops using REINFORCE and on GPU clusters using… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.

1
I built a tiny LLM to demystify how language models work▲915
Built a ~9M param LLM from scratch to understand how they actually work. Vanilla transformer, 60K synthetic conversations, ~130 lines of PyTorch. Trains in 5 min on a free Colab T4. The fish thinks the meaning of life is food. Fork it and swap the personality for your own character.
Apr 2026 · github.com · its alternatives →
2
I RL-trained an agent that trains models with RL (for ~$1.3k)▲107
RL-training an AI agent to RL-train AI agents. Contribute to Danau5tin/ai-trains-ai development by creating an account on GitHub.
Jul 2026 · github.com · its alternatives →
3
Terminal-Bench-RL: Training long-horizon terminal agents with RL▲125
After training calculator agent via RL, I really wanted to go bigger! So I built RL infrastructure for training long-horizon terminal/coding agents that scales from 2x A100s to 32x H100s (~$1M worth of compute!) Without any training, my 32B agent hit #19 on Terminal-Bench leaderboard, beating Stanford's Terminus-Qwen3-235B-A22! With training... well, too expensive, but I bet the results would be good! *What I did*: - Created a Claude Code-inspired agent (system msg + tools) - Built Docker-isolated GRPO training where each rollout gets its own container - Developed a multi-agent…
2025 · github.com · its alternatives →
4
nanochat▲349
Build your won ChatGPT for $100 on a single GPU
2025 · github.com · its alternatives →
5
LeanRL: Fast PyTorch RL with Torch.compile and CUDA Graphs▲53
We're excited to announce that we've open-sourced LeanRL, a lightweight PyTorch reinforcement learning library that provides recipes for fast RL training using torch.compile and CUDA graphs. By leveraging these tools, we've achieved significant speed-ups compared to the original CleanRL implementations - up to 6x faster! Reinforcement learning is notoriously CPU-bound due to the high frequency of small CPU operations. PyTorch's powerful compiler can help alleviate these issues, but comes with its own costs. LeanRL addresses this challenge by providing simple recipes to accelerate your…
2024 · github.com · its alternatives →
6
Tiny-vLLM – high performance LLM inference engine in C++ and CUDA▲205
Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM - jmaczan/tiny-vllm
May 2026 · github.com · its alternatives →
7
GPT‑5.4 mini and nano▲267
Fast and efficient models optimized for coding and subagents
Mar 2026 · openai.com · its alternatives →
8WM
We made glhf.chat – run almost any open-source LLM, including 405B▲161
Try it out! https://glhf.chat/ Hey HN! We’ve been working for the past few months on a website to let you easily run (almost) any open-source LLM on autoscaling GPU clusters. It’s free for now while we figure out how to price it, but we expect to be cheaper than most GPU offerings since we can run the models multi-tenant. Unlike Together AI, Fireworks, etc, we’ll run any model that the open-source vLLM project supports: we don’t have a hardcoded list. If you want a specific model or finetune, you don’t have to ask us for it: you can just paste the Hugging Face link in and…
2024 · glhf.chat · its alternatives →
9
GPT-4.1 in the API▲375
Announcing GPT-4.1, GPT-4.1 mini, & GPT-4.1 nano in the API
2025 · openai.com · its alternatives →
10
Monostate AItraining▲59
Fine-tuning, RL, and inference in one CLI
Dec 2025 · github.com · its alternatives →
11RA
RL Agent that can auto-optimize your LLM prompts▲14
Hey everyone! Along with my team, I've developed a reinforcement learning system that automatically optimizes LLM prompts, complete with a visualization feature to track both prompt structure and learning progress over time. Take a look here: https://nomadic-ml.github.io/nomadic/cookbooks/Nomadic_Promp... Check out our website too:https://www.nomadicml.com/ In terms of how this visualization works: The RL Prompt Optimizer employs a reinforcement learning framework to iteratively improve prompts used for language model evaluations. At each episode, the…
2024 · nomadic-ml.github.io · its alternatives →
12
TuneTrain.ai▲79
Do you know that Small Language Models (SLM) can outperform LLMs if trained (fine-tuned) on domain specific datasets, like your companies knowledge base? The issue: fine-tuning a SLM usually requires a dataset of 10k - 100k records, which is huge. I created a platform for applying "human in the loop" augmentation techniques on your small dataset, so that you can start with maybe 100 records and build-up quickly huge datasets and launch a training without prior knowledge. I implemented 2 techniques, based on LLM distillation, however: more to come. HN, what Do you think? Do you see value in…
2025 · tunetrain.ai · its alternatives →
13
Minimal LLM Post-Training Experiments on an 8GB GPU (SFT, DPO, GRPO)▲21
Minimal, readable LLM post-training experiments on one 8GB GPU. Measures forgetting, seed variance, and RL emergence. - pochenai/nano-llm-posttraining
Aug 2026 · github.com · its alternatives →
14
Next-Gen AI Training: LLM-RLHF-Tuning with PPO and DPO▲30
Comprehensive toolkit for Reinforcement Learning from Human Feedback (RLHF) training, featuring instruction fine-tuning, reward model training, and support for PPO and DPO algorithms with various configurations for the Transformer 5 models including Qwen 3.0 - raghavc/LLM-RLHF-Tuning-with-PPO-and-DP
2024 · github.com · its alternatives →
15TL
Tiny LLMs – Browser-based private AI models for a wide array of tasks▲142
2023 · tinyllms.vercel.app · its alternatives →
16
Dolly▲113
Democratizing the magic of ChatGPT with open models
2023 · github.com · its alternatives →
17
Predibase Reinforcement Fine-Tuning▲175
LLM reinforcement fine-tuning platform to improve LLM output
2025 · rubrik.com · its alternatives →
18IB
In-Browser Graph RAG with Kuzu-WASM and WebLLM▲158
We show the potential of modern, embedded graph databases in the browser by demonstrating a fully in-browser chatbot that can perform Graph RAG using Kuzu (the graph database we're building) and WebLLM, a popular in-browser inference engine for LLMs. The post retrieves from the graph via a Text-to-Cypher pipeline that translates a user question into a Cypher query, and the LLM uses the retrieved results to synthesize a response. As LLMs get better, and WebGPU and Wasm64 become more widely adopted, we expect to be able to do more and more in the browser in combination with LLMs, so a lot of…
2025 · blog.kuzudb.com · its alternatives →
19
Llama-8B Teaches Itself Baby Steps to Deep Research Using RL▲39
I've been tinkering with getting Llama-8B to bootstrap its own research skills through self-play. The model generates questions about documents, searches for answers, and then learns from its own successes/failures through RL (hacked up Unsloth's GRPO code). Started with just 23% accuracy on Apollo 13 mission report questions and hit 53% after less than an hour of training. Everything runs locally using open-source models. It's cool to see the model go from completely botching search queries to iteratively researching to get the right answer.
2025 · github.com · its alternatives →
20
Pipelex – Declarative language for repeatable AI workflows▲122
We’re Robin, Louis, and Thomas. Pipelex is a DSL and a Python runtime for repeatable AI workflows. Think Dockerfile/SQL for multi-step LLM pipelines: you declare steps and interfaces; any model/provider can fill them. Why this instead of yet another workflow builder? - Declarative, not glue code: you state what to do; the runtime figures out how. - Agent-first: each step carries natural-language context (purpose, inputs/outputs with meaning) so LLMs can follow, audit, and optimize. Our MCP server enables agents to run pipelines but also to build new pipelines on demand. - Open…
Oct 2025 · github.com · its alternatives →
21
NanoEuler – GPT-2 scale model in pure C/CUDA from scratch▲55
Hi everyone, I started working on nanoeuler after the ban of anthropic's fable because my ambition and dream is to work in the AI field in anthropic. The two interesting reasons that led me to create nanoeuler were (1) interfacing with llm does not mean understanding how they are composed and (2), working on llm with a very low-level layer to understand the correlation between parameters and data and growth of the model and how the GPU works and how some layers can be optimized. So I started working on it with a research aspect by making nanoeuler grow more and more but doing one step after…
Jun 2026 · github.com · its alternatives →
22
less than 650 LOC trainable GPT only using NumPy▲90
Absolute minimalistic implementation of a GPT-like transformer using only numpy (<650 lines). - joennlae/tensorli
2023 · github.com · its alternatives →
23
Easily train AlphaZero-like agents on any environment you want▲87
Easily train AlphaZero-like agents on any environment you want! - s-casci/tinyzero
2023 · github.com · its alternatives →
24
Create-LLM – Train your own LLM in 60 seconds▲54
https://medium.com/@theaniketgiri/three-months-ago-i-wanted-...
Oct 2025 · github.com · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →