LLMadness – March Madness Model Evals
I wanted to play around with the non-coding agentic capabilities of the top LLMs so I built a model eval predicting the March Madness bracket. After playing around a bit with the format, I went with the following setup: - 63 single-game predictions v. full one-shot bracket - Maxed out at 10 tool calls per game - Upset-specific instruction in the system prompt - Exponential scoring by round (1, 2, 4, 8, 16, 32) There were some interesting learnings: - Unsurprisingly, most brackets are close to chalk. Very few significant upsets were predicted. - There was a HUGE cost and token disparity with…
What it does
In the maker’s words, at launch
I wanted to play around with the non-coding agentic capabilities of the top LLMs so I built a model eval predicting the March Madness bracket. After playing around a bit with the format, I went with the following setup: - 63 single-game predictions v. full one-shot bracket - Maxed out at 10 tool calls per game - Upset-specific instruction in the system prompt - Exponential scoring by round (1, 2, 4, 8, 16, 32) There were some interesting learnings: - Unsurprisingly, most brackets are close to chalk. Very few significant upsets were predicted. - There was a HUGE cost and token disparity with the exact same setup and constraints. Both Claude models spent over $40 to fill in the bracket while MiMo-V2-Flash spent $0.39. I spent a total of $138.69 on all 15 model runs. - There was also a big disparity in speed. Claude Opus 4.6 took almost 2 full days to finish the 2 play-ins and 63 bracket games. Qwen 3.5 Flash took under 10 minutes. - Even when given the tournament year (2026), multiple models pulled in information from previous years. Claude seemed to be the biggest offender, really wanting Cooper Flagg to be on this year's Duke team. This was a really fun way to combine two of my interests and I'm excited to see how the models perform over the coming weeks. You can click into each bracket node to see the full model trace and rationale behind the picks. The stack is Typescript, Next.js, React, and raw CSS. No DB, everything stored in static JSON files. After each game, I update the actual results and re-deploy via GitHub Pages. I wanted to work as fast as possible since the brackets lock today so almost all of the code was AI-generated (shocker). Hope you enjoy checking it out!
Does the same job
all alternatives →- AMApplying Machine Learning to March Madness2019 · github.com · ▲97
- UPUse predictive modeling to win your bracket competition2023 · nflbracket.xyz · ▲5
Hi! I made this as a side project this week to submit a bracket for our friends' NFL playoff bracket competition. It is all html and vanilla JS, runs directly in the browser, and is pretty fun to take for a spin. Obviously it is far from polished - that might have to wait for next year!
- MMMarch Madness for Hackers2014 · codersbracket.com · ▲100
- CTCryptoFinalFour – Tradable March Madness brackets as ERC-721 tokens2018 · cryptofinalfour.com · ▲34
More dev tools this month
the category →
Open-source GTM skills for technical founders
Dev tools · 29d ago · gtmcofounder.com
OpenTrailPaper is open-source bike computer firmware for the LilyGO T5S3 4.7" E-Paper PRO. It supports offline maps, GPX routes, FIT recording and Bluetooth sensors.
Dev tools · 1d ago · opentrailpaper.com
Launched alongside, March 2026
the whole month →
Switch from ChatGPT to Claude with import memory feature
AI · Mar 2026 · claude.com
