PhAIL – Real-robot benchmark for AI models
I built this because I couldn't find honest numbers on how well VLA models [1] actually work on commercial tasks. I come from search ranking at Google where you measure everything, and in robotics nobody seemed to know. PhAIL runs four models (OpenPI/pi0.5, GR00T, ACT, SmolVLA) on bin-to-bin order picking – one of the most common warehouse operations. Same robot (Franka FR3), same objects, hundreds of blind runs. The operator doesn't know which model is running. Best model: 64 UPH. Human teleoperating the same robot: 330. Human by hand: 1,300+. Everything is public – every run with…
PhAIL – Real-robot benchmark for AI models launched on March 31, 2026 with 21 votes, #858 of 23,380 launches that month and more than 90% of that year's launches.
In plain words
PhAIL is a real-world benchmark that measures how well vision-language-action AI models perform on warehouse bin-picking tasks using a Franka FR3 robot. It tests four leading models through hundreds of blind runs with the same objects and setup, providing transparent performance data including video and telemetry for each run. The results show the best-performing model achieving 64 units per hour compared to 330 for a human operator and 1,300+ for manual work, with an open leaderboard accepting new model submissions.
written from the facts on this page · September 2026
From the sources
Four leading models. One commercial task. Production metrics.
In the maker’s words, at launch
I built this because I couldn't find honest numbers on how well VLA models [1] actually work on commercial tasks. I come from search ranking at Google where you measure everything, and in robotics nobody seemed to know. PhAIL runs four models (OpenPI/pi0.5, GR00T, ACT, SmolVLA) on bin-to-bin order picking – one…
Does a similar job
all alternatives →- EFEcho – Fable-level results at 1/3 the cost using open-weight modelsJul 2026 · ▲484
I’ve been building Echo (https://echo.tracerml.ai/), an experiment in making one AI system out of a pool of open-weight models rather than choosing a single model and using it for every task. It started with a simple experiment. I took a group of models, including GLM-5.2, Kimi K2.7 and others, and ran them on the same evaluations. Then I measured what would happen if, for each problem, you somehow knew in advance which models would be useful and how their outputs should be combined. That hypothetical system performed substantially better than any individual model in the pool.…
Coarena by CoastyAug 2026 · coarena.ai · ▲111We built a computer use agent once and got it benchmarked at the 2nd position in the biggest benchmark for this field but in the process, we realized one thing, how outdated and inefficient these benchmarks are and how they don't truly capture how these tasks went. So we thought a community led benchmarking arena would be the way to go about. https://coarena.ai/. You just have to put in a task and two side by side windows with hidden models will show up and do the same task and at the end you vote which one did better(or none). We have a leaderboard where we rank all the…
- PBPac-Bench – How well can models one-shot a Pac-Man game?24d ago · jonclegg.github.io · ▲80
Pac-Man bench: how well a model and harness can recreate Pac-Man from a single prompt.
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 29d ago · world.pixverse.video
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, March 2026
the whole month →
Switch from ChatGPT to Claude with import memory feature
AI · Mar 2026 · claude.com
Littlebird▲709Littlebird remembers everything across your meetings, messages, docs, and apps, then takes action so you can recall any detail and never lose context.
AI · Mar 2026 · littlebird.ai
