Life & fun · alternatives · 2026

24 alternatives to LeanRL: Fast PyTorch RL with Torch.compile and CUDA Graphs

We're excited to announce that we've open-sourced LeanRL, a lightweight PyTorch reinforcement learning library that provides recipes for fast RL training using torch.compile and CUDA graphs. By leveraging these tools, we've achieved significant speed-ups compared to the original CleanRL implementations - up to 6x faster! Reinforcement learning is notoriously CPU-bound due to the high frequency of small CPU operations. PyTorch's powerful compiler can help alleviate these issues, but comes with its own costs. LeanRL addresses this challenge by providing simple recipes to accelerate your…

LeanRL is a lightweight PyTorch reinforcement learning library offering recipes for accelerated RL training using torch.compile and CUDA graphs. It addresses the CPU bottleneck common in reinforcement learning by… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. LeanRL: Fast PyTorch RL with Torch.compile and CUDA Graphs launched in 2024; newer entries below may have overtaken it.

1
I stripped DALL·E Mini to its bare essentials and converted it to Torch▲663
min(DALL·E) is a fast, minimal port of DALL·E Mini to PyTorch - kuprel/min-dalle
2022 · github.com · its alternatives →
2
Forge Agent▲113
Swarm Agents That Turn Slow PyTorch Into Fast GPU Kernels
Jan 2026 · rightnowai.co · its alternatives →
3
Terminal-Bench-RL: Training long-horizon terminal agents with RL▲125
After training calculator agent via RL, I really wanted to go bigger! So I built RL infrastructure for training long-horizon terminal/coding agents that scales from 2x A100s to 32x H100s (~$1M worth of compute!) Without any training, my 32B agent hit #19 on Terminal-Bench leaderboard, beating Stanford's Terminus-Qwen3-235B-A22! With training... well, too expensive, but I bet the results would be good! *What I did*: - Created a Claude Code-inspired agent (system msg + tools) - Built Docker-isolated GRPO training where each rollout gets its own container - Developed a multi-agent…
2025 · github.com · its alternatives →
4
Rlpyt – Modular, Optimized Deep Reinforcement Learning in PyTorch▲7
Reinforcement Learning in PyTorch. Contribute to astooke/rlpyt development by creating an account on GitHub.
2019 · github.com · its alternatives →
5
Supercharging RL with Hyper-Efficient Online Opt, +165% in 2h, $10▲32
Arc enables continual learning for agents to adapt, learn, and transfer knowledge in live environments. Open-source SDK for building agents that learn from experience and improve over time.
2025 · arc.computer · its alternatives →
6
I RL-trained an agent that trains models with RL (for ~$1.3k)▲107
RL-training an AI agent to RL-train AI agents. Contribute to Danau5tin/ai-trains-ai development by creating an account on GitHub.
Jul 2026 · github.com · its alternatives →
7
Fast Deep Reinforcement Learning Course▲148
I worked on this applied Deep Reinforcement Learning course for the better part of 2021. I made a Datacamp course [0] before, and this served as my inspiration to make an applied Deep RL series. Normally, Deep RL courses teach a lot of mathematically involved theory. You get the practical applications near the end (if at all). I have tried to turn that on its head. In the top-down approach, you learn practical skills first, then go deeper later. This is much more fun. This course (the first in a planned multi-part series) shows how to use the Deep Reinforcement Learning framework RLlib to…
2022 · courses.dibya.online · its alternatives →
8
Zipslicer, a library for loading LLM checkpoints on consumer hardware▲38
This is a low-level opensource library I developed for my own use and decided to share, as it makes it possible to process large checkpoints of neural networks without renting high-RAM instances, on a regular PC. It replaces torch.load() with a custom function that produces a dictionary that materializes tensors on the fly. Compared to other solutions it doesn't require sharding or re-encoding checkpoints and uses them completely as-is. It is a foundation to make it possible to run inference and compress language models and other large models one layer at a time - in principle, even one…
2023 · github.com · its alternatives →
9
NanoRL – RL training for LLMs in ~1,800 lines▲11
The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster (e.g. 8xH100 trainer, 8 vLLM workers, ran as a [SkyPilot job group](https://docs.skypilot.ai/en/latest/examples/job-groups.html) on k8s ). All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.
Aug 2026 · github.com · its alternatives →
10
I built a tensor library from scratch in C++/CUDA▲119
Hi HN, Over the past few months, I've been building `dsc`, a tensor library from scratch in C++/CUDA. My main focus has been on getting the basics right, prioritizing a clean API, simplicity, and clear observability for running small LLMs locally. The key features are: - C++ core with CUDA support written from scratch. - A familiar, PyTorch-like Python API. - Runs real models: it's complete enough to load a model like Qwen from HuggingFace and run inference on both CUDA and CPU with a single line change[1]. - Simple, built-in observability for both Python and C++. Next on the roadmap is…
2025 · github.com · its alternatives →
11
Keep your PyTorch model in VRAM by hot swapping code▲77
Pytorch script hot swap: Change code without unloading your LLM from VRAM - valine/training-hot-swap
2025 · github.com · its alternatives →
12BA
Brancher – A user-friendly PyTorch module for probabilistic inference▲96
2019 · brancher.org · its alternatives →
13
ART – a new open-source RL framework for training agents▲116
Hey HN, I wanted to share a new project we've been working on for the last couple of months called ART (https://github.com/OpenPipe/ART). ART is a new open-source framework for training agents using reinforcement learning (RL). RL allows you to train an agent to perform better at any task whose outcome can be measured and quantified. There are many excellent projects focused on training LLMs with RL, such as GRPOTrainer (https://huggingface.co/docs/trl/main/en/grpo_trainer) and verl…
2025 · github.com · its alternatives →
14
A weekend replication of STOKE, a stochastic superoptimiser▲58
Haskell re-implementation of STOKE, the stochastic superoptimizer - blaze/notebooks/tutorial.ipynb at master · bollu/blaze
2019 · github.com · its alternatives →
15
Comgra: A library for debugging and understanding neural networks▲66
I'm a machine learning engineer and researcher. I got fed up with how difficult it is to understand why neural networks behave the way they do, so i wrote a library to help with it. Comgra (computation graph analysis) is a library you can use with pytorch to extract all the tensor data you care about and visualize it graphically in a browser. This allows for a much more detailed analysis of what is happening than the usual approach of using tensorboard. You can go investigate tensors as training proceeds, drill down into individual neurons, inspect single data sets that are of special…
2023 · github.com · its alternatives →
16
TorchGAN – Research Framework for Modelling GANs Based on Pytorch▲56
Research Framework for easy and efficient training of GANs based on Pytorch - torchgan/torchgan
2018 · github.com · its alternatives →
17
GUIs for Faster ML Prototyping and Sharing▲85
Build and share delightful machine learning apps, all in Python. 🌟 Star to support our work! - gradio-app/gradio
2020 · github.com · its alternatives →
18
Keys and Caches▲66
Vibe profile your ML models to get max performance.
2025 · github.com · its alternatives →
19
PyTorch Lightning▲79
The light PyTorch wrapper for high-performance AI research
2020 · pytorchlightning.ai · its alternatives →
20
FlashML▲15
Drag & Drop Machine Learning Models w/ PyTorch Flexibility.
2020 · its alternatives →
21
Note_rl▲1
Reinforcement learning library for Keras and PyTorch.
2025 · github.com · its alternatives →
22
Torchmtl a pytorch library for multi-task learning▲6
A lightweight module for Multi-Task Learning in pytorch. - chrisby/torchMTL
2020 · github.com · its alternatives →
23
Graphlearn-for-PyTorch, distributed graph learning on PyTorch▲9
Hello HN, We are pleased to introduce you graphlearn-for-pytorch (https://github.com/alibaba/graphlearn-for-pytorch), an open-source distributed graph neural network library based on PyTorch and compatible with PyG. Our library is designed to make it easy for developers to build and train large-scale graph models in a distributed environment. With graphlearn-for-pytorch, you can leverage GPUs to accelerate graph sampling and utilize UVA to reduce the overheads of feature collection. Following a scalable design, graphlearn-for-pytorch supports training GNN models on…
2023 · github.com · its alternatives →
24RA
RL Agent that can auto-optimize your LLM prompts▲14
Hey everyone! Along with my team, I've developed a reinforcement learning system that automatically optimizes LLM prompts, complete with a visualization feature to track both prompt structure and learning progress over time. Take a look here: https://nomadic-ml.github.io/nomadic/cookbooks/Nomadic_Promp... Check out our website too:https://www.nomadicml.com/ In terms of how this visualization works: The RL Prompt Optimizer employs a reinforcement learning framework to iteratively improve prompts used for language model evaluations. At each episode, the…
2024 · nomadic-ml.github.io · its alternatives →

Also compare

Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →