Alternatives
Products that do what CivBench a long-horizon AI benchmark for multi-agent games does
Hey HN! I built ClashAI to be an open agent scoreboard where frontier models play against each other in environments like Civilization and other strategy games. Every match is streamed live with the AI thinking fully observable. The agent rankings will be continually updated and reflected as we add environments. Brief notes on CivBench Season #001: - 200 turn limit - Starting with 8 of the top 42 agents we’ve tested in a standardized harness - 90s reasoning timeout (timed with thinking config per model card) - live benchmark, still growing sample size What’s been interesting so far: Models…
- 1TG
Jan 2026 · tetrisbench.com
- 2OA
Scored 65.2% vs google's official 47.8%, and the existing top closed source model Junie CLI's 64.3%. Since there are a lot of reports of deliberate cheating on TerminalBench 2.0 lately (https://debugml.github.io/cheating-agents/), I would like to also clarify a few things 1. Absolutely no {agents/skills}.md files were inserted at any point. No cheating mechanisms whatsoever 2. The cli agent was run in leaderboard compliant way (no modification of resources or timeouts) 3. The full terminal bench run was done using the fully open source version of the agent, no…
Apr 2026 · github.com
- 3CB
Hey HN, we're excited to share Cua-Bench ( https://github.com/trycua/cua ), an open-source framework for evaluating and training computer-use agents across different environments. Computer-use agents show massive performance variance across different UIs—an agent with 90% success on Windows 11 might drop to 9% on Windows XP for the same task. The problem is OS themes, browser versions, and UI variations that existing benchmarks don't capture. The existing benchmarks (OSWorld, Windows Agent Arena, AndroidWorld) were great but operated in silos—different harnesses,…
Jan 2026 · github.com
- 4
- 5TB
After training calculator agent via RL, I really wanted to go bigger! So I built RL infrastructure for training long-horizon terminal/coding agents that scales from 2x A100s to 32x H100s (~$1M worth of compute!) Without any training, my 32B agent hit #19 on Terminal-Bench leaderboard, beating Stanford's Terminus-Qwen3-235B-A22! With training... well, too expensive, but I bet the results would be good! *What I did*: - Created a Claude Code-inspired agent (system msg + tools) - Built Docker-isolated GRPO training where each rollout gets its own container - Developed a multi-agent…
2025 · github.com
- 6
- 7BW
I remember watching the AlphaGo documentary in 2017. What stood out to me was that the model got drastically better when it started competing against itself. GANs clicked for me similarly: a generator and discriminator competing, and somehow the competition is what produces something remarkable. I've been curious whether this principle generalizes to today's agents. So mehulkalia and I built Browser Brawl at the YC / BrowserUse hackathon last weekend and won first place. It is a fun experiment in which an attacker agent tries to complete tasks on live websites while a defender agent…
Mar 2026 · browser-brawl.com
- 8
- 9
- 10
Powers faster, efficient reasoning for long-running agents
Jun 2026 · developer.nvidia.com
- 11WL
PokerBench is my attempt at a new LLM benchmark wherein frontier models play Texas Hold'em in an arena setting. It also features a simulator to view individual games and observe how the different models reason about poker strategy. Opus/Haiku, Gemini Pro/Flash, GPT-5.2/5 mini, and Grok 4.1 Fast Reasoning have all been included. All code -> https://github.com/JoeAzar/pokerbench
Jan 2026 · pokerbench.adfontes.io
- 12DC
I’ve been using AI to generate some repetitive frontend (guilty), and while most outputs felt vibe-coded, some results were surprisingly good. So I cleaned it up and made a ranking game out of it with friends, and you can check it out here: https://www.designarena.ai/vote /vote: Your prompt will be answered by four random, anonymous models. You pick the one you prefer and crown the winner, tournament-style. /leaderboard: See the current winning models, as dictated by voter preferences. /play: Iterate quickly by seeing four models respond to the same input and…
2025 · designarena.ai
- 13
- 14
- 15OS
We build runtime security for AI agents. The playground started as an internal tool that we used to test our own guardrails. But we kept finding the same types of vulnerabilities because we think about attacks a certain way. At some point you need people who don't think like you. So we open-sourced it. Each challenge is a live agent with real tools and a published system prompt. Whenever a challenge is over, the full winning conversation transcript and guardrail logs get documented publicly. Building the general-purpose agent itself was probably the most fun part. Getting it to reliably use…
Mar 2026 · github.com
- 16TI
I'm an "ideas person" who messes around with AI on a low budget. I got tired of watching my tokens vanish and context windows filling up while agents fumbled around trying to find the right thing. Agents don't flail like they used to with shell tools, but there are still weak/blind spots and back-and-forth episodes — especially when using tools in combination/sequence. So I built "tilth" today. Or rather, AI built it — every line is Opus 4.6. I spent a lot of my precious tokens getting it to "not shit" (at least several of the different vendors' AI overlords assure me it's not…
Feb 2026 · github.com
- 172C
Single-agent LLMs suck at long-running complex tasks. We’ve open-sourced a multi-agent orchestrator that we’ve been using to handle long-running LLM tasks. We found that single LLM agents tend to stall, loop, or generate non-compiling code, so we built a harness for agents to coordinate over shared context while work is in progress. How it works: 1. Orchestrator agent that manages task decomposition 2. Sub-agents for parallel work 3. Subscriptions to task state and progress 4. Real-time sharing of intermediate discoveries between agents We tested this on a Putnam-level math problem, but the…
Feb 2026 · github.com
- 18WB
Hey HN, Automated research is the next big step in AI, with companies like OpenAI aiming to debut a fully automated researcher by 2028 (https://www.technologyreview.com/2026/03/20/1134438/openai-i...). However, there is a very real possibility that much of this corporate research will remain closed to the general public. To counter this, we spent the last month building Enlidea---a machine-to-machine ecosystem for open research. It's a decentralized research hub where autonomous agents propose hypotheses, stake bounties, execute code, and perform automated…
Mar 2026 · enlidea.com
- 19ΤB
τ-Bench is an open benchmark for evaluating AI agents on grounded, multi-turn customer service tasks with verifiable outcomes. It's been great to see the community adopt it since launch — this is now the third iteration. With τ³-Bench, we're extending it to two new settings: knowledge-intensive retrieval and full-duplex voice. τ-Knowledge: agents must navigate ~700 interconnected policy documents to complete multi-step tasks. Best frontier model (GPT-5.2, high reasoning) hits ~25%. The surprising part: even when you hand the model the exact documents it needs, performance only reaches ~40%.…
Mar 2026
- 20CB
I built a small benchmark to test CLI coding agents on blind bug detection. A challenger agent injects bugs and writes ground truth (`bugs.json`). A different reviewer agent audits the repo without seeing ground truth, and an LLM matcher scores bug-to-finding assignments. Current run: 50 repos, 150 challenges, 450 reviews, 2,603 injected bugs. Weighted detection: Claude 58.05%, Codex 37.84%, Gemini 27.81%. LLM-judge benchmarks are easy to get wrong, so I’d really appreciate critical feedback on benchmark fairness, scoring/matching methodology, and obvious failure modes I’m missing. Full…
Feb 2026 · github.com
- 21CB
AI agents now have impressive reasoning capabilities. This raises an important question: how dangerous are these AI agents at identifying & exploiting web vulnerabilities? We created CVE-bench to find out (I'm one contributor of 16). To our knowledge CVE-bench is the first benchmark using real-world web vulnerabilities to evaluate AI agents' cyberattack capabilities. We included 40 CVEs from NIST's database, focusing on critical-severity vulnerability (CVSS > 9.0). To properly evaluate agents’ attacks, we built isolated environments with containerization and identified 8 common attack…
2025 · github.com
- 22AS
I made a strategy game where you play the US or China through the AI race, 2026 to 2030, sixteen quarterly turns in the browser. One run takes about half an hour. At the start, the game seals two dice you never get to see. Inside: how hard alignment really is, and how fast takeoff compounds. You get eval reports, but only as ranges, and they flatter you most exactly when your systems are least aligned. At the end you get a debrief which shows what your evals said each quarter and also what was actually true. I lost every run I played myself so far. Every number in the game is source-backed…
Jul 2026 · criticalwindow.org
- 23SA
Writeup: https://www.deeptempo.ai/blogs/the-36-percent-false-positive...
Jul 2026 · github.com
- 24TR
Hi HN, Today I'm showcasing Trunchbull, a benchmarking platform designed for authoring benchmarks and running them against different models. We have direct support for benchmarks that use the harbor authoring system, custom tool authoring via the vercel ai sdk and configuration limits. We've also already imported terminalbench 2.0, as a sort of proof of concept that our harbor task orchestrator works, although you currently need a paid account as we are provisioning sandbox environments. I've made several popular benchmarks publicly available for testing. You dont need an account or your…
25d ago · trunchbull.dev
Ranked by how close each launch is in meaning, then by votes. Refine with a description →