Chess-LLM, using constrained-generation to force LLMs to battle it out
As I was playing with the Outlines library (https://outlines-dev.github.io/outlines/), I discussed with my friend Maxime how funny it would be if we set up a way to pair LLMs in chess matches till one wins. The first time I tried it, it required substantial prompt engineering to get some of those LLMs to propose valid moves. Large language models can mostly stay focused and even play rather well; see https://news.ycombinator.com/item?id=37616170 for example. However small language models aren't as easy to convince. Some of those LLMs have seen very little…
What it does
In the maker’s words, at launch
As I was playing with the Outlines library (https://outlines-dev.github.io/outlines/), I discussed with my friend Maxime how funny it would be if we set up a way to pair LLMs in chess matches till one wins. The first time I tried it, it required substantial prompt engineering to get some of those LLMs to propose valid moves. Large language models can mostly stay focused and even play rather well; see https://news.ycombinator.com/item?id=37616170 for example. However small language models aren't as easy to convince. Some of those LLMs have seen very little chess notation and so after the first few opening moves there aren't any valid tactics, let alone strategy, so they would end up either repeating the same move, or hallucinate moves that are not valid (Kxe5, but there would be a queen on e5!) Then Outlines came along and we could force them to pick valid moves with little cost! Maxime worked super fast and got a first version of this idea as a gradio space. I think it is pretty fun to see the (mostly terrible, but otherwise valid) chess that those LLMs play. Maybe it will even be instructive to how we can create small LLMs that can play much better than the ones on the leaderboard. Anyway, you can check it out here: https://huggingface.co/spaces/mlabonne/chessllm What is interactive about it: you can pick the LLMs from available models on HuggingFace (within reason, small LLMs are preferable so that the space does not crash) or push one of your own small models to HF and have it fight with others. At the end of the game the leaderboard is updated. Hope you find it fun!
Does the same job
all alternatives →
- LALLM Arena – a text-based game that explores the possibilities of LLMs2023 · llmarena.com · ▲5
The initial idea for the game came during the final day of Game AI school in Cambridge. There, we had a Jam where we explored the idea of using LLMs as a game engine for fights. We then built a full web version in just a week. There is no need to register or pay to play. Test it out!
- PEPrompt Engineering Chess2023 · github.com · ▲7
Test your prompt engineering skills by writing prompts and battling them against each other! Your prompt template can access the board state, the move history, and a list of legal moves, and the game engine selects the first legal move in the string response from the LLM you query. My best prompt so far ignores the board state and the move history and just tries to play mates, make captures, and promote pawns. Can you do better?
- PPPencil Puzzle Bench – LLM Benchmark for Multi-Step Verifiable ReasoningMar 2026 · ppbench.com · ▲5
I've been working on applying LLMs to long-context, verifiable problems over the past year, and today I'm releasing a benchmark of 62,000 pencil puzzles across 94 types (sudoku, nonori, slitherlink, etc.). The benchmark also allows for intermediate checks /rule breaks for all varieties at any step. I tested 51 models against a subset (300 puzzles) in two modes: single-shot (output the full solution) and agentic (iterate with verifier feedback). Some results: - Best model (GPT 5.2@xhigh) solves 56%. (~ half the puzzles are unsolved by any model) - Agentic solves average 29 turns. The…
- ADAI Debate Arena – See Which LLM Argues Best2025 · bot-bicker.vercel.app · ▲5
Ever wish you could get the best arguments for both sides of a debate? I built an AI-powered debate platform that pits language models against each other on controversial topics. Each AI is randomly assigned a side (pro/con). You vote before and after to see if you were persuaded. Most content today presents lopsided arguments. They provide strong points for one side, weak ones for the other. This project aims to surface the strongest arguments from both sides, using LLMs to simulate a fair debate. With enough usage, I want to use it to benchmark LLMs. My hypothesis is that randomly…
- CACan a person's chess mind outlive them? An attempt with 41 mastersApr 2026 · playchessgate.com · ▲5
Two years ago I sat down to build a chess engine for my kids' Christmas present. Wanted it to play badly enough to be beatable but not randomly bad: different styles so they could choose what to face. While building those styles I noticed the choices started looking like actual historical players' tendencies. Went looking. Turns out: given enough games from one person, you can extract enough of their decision pattern to reproduce it. Two years later, 41 players, ~3,000 games each. Tal sacrifices unsoundly. Morphy attacks like it's 1850. Capablanca grinds endgames. Same engine underneath,…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, March 2024
the whole month →
- 3Y
Life & fun · 2024 · github.com
Microlaunch▲1,116Launch and get feedback on both the idea and product
Dev tools · 2024 · microlaunch.net
