Terminal-Bench-RL: Training long-horizon terminal agents with RL
After training calculator agent via RL, I really wanted to go bigger! So I built RL infrastructure for training long-horizon terminal/coding agents that scales from 2x A100s to 32x H100s (~$1M worth of compute!) Without any training, my 32B agent hit #19 on Terminal-Bench leaderboard, beating Stanford's Terminus-Qwen3-235B-A22! With training... well, too expensive, but I bet the results would be good! *What I did*: - Created a Claude Code-inspired agent (system msg + tools) - Built Docker-isolated GRPO training where each rollout gets its own container - Developed a multi-agent…
In plain words
Terminal-Bench-RL is an infrastructure for training reinforcement learning agents to perform complex terminal and coding tasks over long horizons. It uses Docker-isolated training environments and a hybrid reward system combining unit test verification with LLM-based behavioral evaluation. The system scales from small GPU setups to large clusters and is designed for researchers and developers building autonomous agents capable of executing multi-step programming tasks. A 32-billion-parameter agent trained with this system achieved competitive results on the Terminal-Bench leaderboard without additional training.
written from the facts on this page · September 2026
From the sources
In the maker’s words, at launch
After training calculator agent via RL, I really wanted to go bigger! So I built RL infrastructure for training long-horizon terminal/coding agents that scales from 2x A100s to 32x H100s (~$1M worth of compute!) Without any training, my 32B agent hit #19 on Terminal-Bench leaderboard, beating Stanford's Terminus-Qwen3-235B-A22! With training... well, too expensive, but I bet the results would be good! *What I did*: - Created a Claude Code-inspired agent (system msg + tools) - Built Docker-isolated GRPO training where each rollout gets its own container - Developed a multi-agent synthetic data pipeline to generate & validate training data with Opus-4 - Implemented a hybrid reward signal of unit test verifiers & a behavioural LLM judge. *Key results*: - My untrained Qwen3-32B agent achieved 13.75% on Terminal-Bench (#19, beats Stanford's Qwen3-235B MoE) - I tested training to work stably on 32x H100s distributed across 4 bare metal nodes - I created a mini-eval framework for LLM-judge performance. Sonnet-4 won. - ~£30-50k needed for full training run of 1000 epochs (I could only afford testing ) *Technical details*: - The synthetic dataset ranges from easy to extremely hard tasks. An example hard task's prompt: "I found this mystery program at `/app/program` and I'm completely stumped. It's a stripped binary, so I have no idea what it does or how to run it properly. The program seems to expect some specific input and then produces an output, but I can't figure out what kind of input it needs. Could you help me figure out what this program requires?" - Simple config presets allow training to run on multiple hardware setups with minimal effort. - GRPO used with 16 rollouts per task, up to 32k tokens per rollout. - Agent uses XML/YAML format to structure tool calls *More details*: My Github repos open source it all (agent, data, code) and has way more technical details if you are interested!: - Terminal Agent RL repo - Multi-agent synthetic data pipeline repo I thought I would share this because I believe long-horizon RL is going to change everybody's lives, and so I feel it is important (and super fun!) for us all to share knowledge around this area, and also have enjoy exploring what is possible. Thanks for reading! Dan (Built using rLLM RL framework which was brilliant to work with, and evaluated and inspired by the great Terminal Bench benchmark)
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 27d ago · cactuscompute.com
Launched alongside, July 2025
the whole month →
Trickle - Magic Canvas ▲1,064The 1st Agentic Canvas for building apps visually with AI
AI · 2025 · trickle.so
- TB
Life & fun · 2025 · trytender.app
- TY
Today marks ten years, 3653 consecutive days, of running at least one mile every day under the USRSA rules [1]. To celebrate, I built an interactive dashboard that turns a decade of GPX files into charts you can explore. Running has truly changed my life: I've made lifelong friends, explored beautiful places, and more importantly invested into my own health and fitness, which I'm starting to see the positive benefits as I get older. The stack is pretty simple: a NextJS app, with a Postgres database to keep all my running data, and all the stats are pre-computed and cached in Redis, so I…
Dev tools · 2025 · nodaysoff.run
- DA
Made this website as an exercise in vibe-coding and GCP. It was posted about a few times around the internet, on sites like Morning Brew (https://www.morningbrew.com/issues/business-buzzkill), MetaFilter (https://www.metafilter.com/209703/Draw-A-Fish), boingboing.net, etc. I think it's cute! I built a basic CNN trained against penises and swastikas, and then anything that doesn't hit the 63% confidence score gets sent to a mod queue, a vibe-coded fish-tinder…
Dev tools · 2025 · drawafish.com
