New eval from SWE-bench team evalutes LMs based on goals not tickets
Current evals test LMs on tasks: "fix this bug," "write a test" But we code to achieve goals: maximize revenue, cut costs, win users Meet CodeClash: LMs compete via their codebases across multi-round tournaments to achieve high-level goals. Because real software dev isn’t about following instructions. It’s about achieving outcomes. Here's how it works: Two LMs enter a tournament. Each maintains its own codebase. Every round: 1. Edit Phase: LMs modify their codebases however they like 2. Competition phase: Codebases battle in an arena. 3. Repeat The LM that wins the majority of rounds is…
New eval from SWE-bench team evalutes LMs based on goals not tickets launched on November 5, 2025 with 5 votes, #4,178 of 9,795 launches that month and more than 75% of that year's launches.
What it does
CodeClash: Benchmarking Goal-Oriented Software Engineering
No explicit GitHub issues or tasks. Given just a high-level objective, models decide for themselves what to build. Models evolve their codebases across multiple rounds, analyze gigabytes of logs, adapt strategies, implement algorithms, and make all high- to low-level decisions. Models compete via their codebases in arenas where success is measured by relative scores like income, territory control, survival. But real software development isn't a series of isolated tasks. Improve user retention, increase revenue, reduce costs. We build to achieve outcomes, not to close tickets. What if AI evaluations reflected this dynamism of real-world software development? In CodeClash, models build and…from codeclash.ai
In the maker’s words, at launch
Current evals test LMs on tasks: "fix this bug," "write a test" But we code to achieve goals: maximize revenue, cut costs, win users Meet CodeClash: LMs compete via their codebases across multi-round tournaments to achieve high-level goals. Because real software dev isn’t about following instructions. It’s about…
Does a similar job
all alternatives →
Stage – Putting humans back in control of code reviewApr 2026 · stagereview.app · ▲130Hey HN! We're Charles and Dean, and we're building Stage: a code review tool that guides you through reading a PR step by step, instead of piecing together a giant diff. Here's a demo video: https://www.tella.tv/video/stage-demo-1pph. You can play around with some example PRs here: https://stagereview.app/explore. Teams are moving faster than ever with AI these days, but more and more engineers are merging changes that they don't really understand. The bottleneck isn't writing code anymore, it's reviewing it. We're two engineers who got frustrated with…
- RSRandomly switching between LMs at every step boosts SWE-bench score2025 · swebench.com · ▲5
Hello from the SWE-bench/SWE-agent team at Princeton/Stanford. When we created the SWE-bench benchmark in 2023 from hundreds of real-life GitHub issues/pull requests, the highest score was just a couple of percent. The tasks were so challenging for LMs, that most people didn't even want to work on them. Half a year later, SWE-agent showed that the early 2024 LMs were actually good enough to resolve up to 20% of the GitHub issues in the benchmark. This kicked off a whole wave of coding agents. Back then, developing agents was all about working around tons of silly behavior from…
Improve LLM Performance by Maximizing Iterative Development2024 · github.com · ▲104I have been working in AI space for a while now, first at FAANG with ML since 2021, then with LLM in start-ups since early 2023. I think LLM Application development is extremely iterative, more so than any other types of development. This is because to improve an LLM application performance (accuracy, hallucinations, latency, cost), you need to try various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, different agent architecture, and more. There are thousands of possible combinations and you need a process…
LLMs suck at writing integration code… for now2025 · github.com · ▲20Hi HN! Stefan here from superglue and today I’d like to share a new benchmark we’ve just open sourced: an Agent-API Benchmark, in which we test how well LLMs handle APIs. We gave LLMs API documentation and asked them to write code that makes actual API calls. Things like "create a Stripe customer" or "send a Slack message". We're not testing if they can use SDKs; we're testing if they can write raw HTTP requests (with proper auth, headers, body formatting) that actually work when executed against real API endpoints and can extract relevant information from that response. tl:dr: LLMs suck at…
Stage CLI – An easier way of reading your AI generated changes locallyMay 2026 · github.com · ▲46Hey HN! We're Charles and Dean. A few weeks ago we posted about Stage, a code review tool that guides you through reading a PR step by step - https://news.ycombinator.com/item?id=47796818. We got a lot of great feedback but also heard from many people that they wanted to have the chapters experience even before opening a PR… so we built the Stage CLI as the local, open-source version that anyone can try. Here’s a quick demo video: https://www.tella.tv/video/stage-cli-demo-f55q It works with any coding agent of your choice. The skill instructs the agent to…
More dev tools this month
the category →
Contribute to oosexgang/hackios-xnu-kernel development by creating an account on GitHub.
Dev tools · 2d ago · github.com
open-source IDE for thoughtful software design. Contribute to devdotfast/whiteboard development by creating an account on GitHub.
Dev tools · 30d ago · github.com
Contribute to reladraw/reladraw development by creating an account on GitHub.
Dev tools · 28d ago · github.com
OpenBot▲362Grok Bot alternative: free, local, open-source, multiplayer
Dev tools · 25d ago · openbot.run
Talanoa▲346Hi HN, I’m a fullstack developer, and honestly, I’ve lost count of how many side projects I’ve worked on over the years. But one idea kept coming back to me: a different kind of email client. It first came to me in 2014: “What if we could handle emails more like conversations?” Every year, I’d revisit this concept in my “Build Notes.” Ten years later, no one had built exactly what I had in mind — so I finally decided to do it myself. That’s how Talanoa was born. I wanted something focused on people, not timelines. I was tired of marking messages as unread just to remember to reply later, or…
Dev tools · 22d ago · talanoa.email
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, November 2025
the whole month →
Guideflow▲1,338Create interactive product demos and step-by-step guides in seconds with Guideflow, the leading AI-powered demo automation tool.
AI · Nov 2025 · guideflow.com
Hardware and software projects by Ali Roberts
Life & fun · Nov 2025 · bitsnpieces.dev
Welltory▲1,028Measure your wellness. Use data to feel great every day.
Work · Nov 2025 · welltory.com
TrustMRR▲833LLM Gateway is a b2b artificial intelligence based in United States, founded in 2025. $259,539/month in verified Stripe revenue. $78,764 MRR. Ahrefs Domain Rating 52/100. growing 55% month-over-month. Track LLM Gateway's verified metrics on TrustMRR.
Growth · Nov 2025 · trustmrr.com
Boing▲782Life & fun · Nov 2025 · boing.greg.technology