Work · alternatives · 2026
24 alternatives to New Benchmark from SWE-bench team is 0% solved
ProgramBench evaluates whether language models can rebuild programs from scratch.
New Benchmark is a tool from the SWE-bench team designed for software engineers and researchers working on code generation and problem-solving tasks. It provides a testing framework with unsolved problems that… Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes.
- 1
SWE-Kit▲568Build your own Devin like software engineering agent
2024 · swekit.dev · its alternatives →
- 2
PinchBench▲305Find the best AI model for your OpenClaw
Mar 2026 · pinchbench.com · its alternatives →
- 3
- 4
PHBench▲398Predict the next Series A from a ProductHunt launch
May 2026 · phbench.com · its alternatives →
- 5
Hey Folks! I've been building an open source benchmark for measuring local LLM performance on your own hardware. The benchmarking tool is a CLI written on top of Llamafile to allow for portability across different hardware setups and operating systems. The website is a database of results from the benchmark, allowing you to explore the performance of different models and hardware configurations. Please give it a try! Any feedback and contribution is much appreciated. I'd love for this to serve as a helpful resource for the local AI community. For more check out: - Website:…
2025 · localscore.ai · its alternatives →
- 6RS
Hello from the SWE-bench/SWE-agent team at Princeton/Stanford. When we created the SWE-bench benchmark in 2023 from hundreds of real-life GitHub issues/pull requests, the highest score was just a couple of percent. The tasks were so challenging for LMs, that most people didn't even want to work on them. Half a year later, SWE-agent showed that the early 2024 LMs were actually good enough to resolve up to 20% of the GitHub issues in the benchmark. This kicked off a whole wave of coding agents. Back then, developing agents was all about working around tons of silly behavior from…
2025 · swebench.com · its alternatives →
- 7
Hey HN! We're Charles and Dean, and we're building Stage: a code review tool that guides you through reading a PR step by step, instead of piecing together a giant diff. Here's a demo video: https://www.tella.tv/video/stage-demo-1pph. You can play around with some example PRs here: https://stagereview.app/explore. Teams are moving faster than ever with AI these days, but more and more engineers are merging changes that they don't really understand. The bottleneck isn't writing code anymore, it's reviewing it. We're two engineers who got frustrated with…
Apr 2026 · stagereview.app · its alternatives →
- 8WH
2022 · memgraph.com · its alternatives →
- 9
Cognition's coding model, 64% cheaper than Fable 5.1
2024 · cognition.com · its alternatives →
- 10
We’re excited to share ML-Dev-Bench, a new open-source benchmark that tests AI agents on real-world ML development tasks. Unlike typical coding challenges or Kaggle-style competitions, our benchmark simulates end-to-end ML workflows including: - Dataset handling and preprocessing - Debugging model and code failures - Implementing new model architectures - Fine-tuning and improving existing models With 30 diverse tasks, ML-Dev-Bench evaluates agents across critical stages of ML development. To complement this, we built Calipers, a framework that provides systematic performance evaluation and…
2025 · github.com · its alternatives →
- 11
PromptForge▲474PromptForge: Open-source AI prompt engineering workbench with systematic evaluation I built PromptForge after getting frustrated with the manual trial-and-error approach to prompt engineering. Most tools are just text editors, but I wanted something that brings software engineering discipline to prompt development. Key technical features: - AI-assisted prompt generation (stop starting from scratch) - Systematic evaluation engine that auto-generates test suites - Multi-model support (Claude, GPT-4, Azure OpenAI) - Go backend with SQLite for performance and simplicity - One-line Docker…
2025 · github.com · its alternatives →
- 12
Middleware▲257Open-source DORA metrics for software engineering teams
2024 · github.com · its alternatives →
- 13
Weave▲237Hey HN, We’re building Weave: an ML-powered tool to measure engineering output, that actually understands engineering output! Why? Here’s the thing: almost every eng leader already measures output - either openly or behind closed doors. But they rely on metrics like lines of code (correlation with effort: ~0.3), number of PRs, or story points (slightly better at ~0.35). These metrics are, frankly, terrible proxies for productivity. We’ve developed a custom model that analyzes code and its impact directly, with a far better 0.94 correlation. The result? A standardized engineering output…
2025 · app.workweave.ai · its alternatives →
- 14
APIEval-20▲121An open benchmark for AI agents that test APIs
May 2026 · resources.kusho.ai · its alternatives →
- 15
A generic latency benchmarking library. Contribute to tylertreat/bench development by creating an account on GitHub.
2015 · github.com · its alternatives →
- 16
- 17
Deepmark AI▲97LLM benchmarking tool for task-specific metrics on your data
2023 · github.com · its alternatives →
- 18
Hi HN! Erik here from Pig.dev, and today I'd like to share a new project we've just open sourced: Muscle Mem is an SDK that records your agent's tool-calling patterns as it solves tasks, and will deterministically replay those learned trajectories whenever the task is encountered again, falling back to agent mode if edge cases are detected. Like a JIT compiler, for behaviors. At Pig, we built computer-use agents for automating legacy Windows applications (healthcare, lending, manufacturing, etc). A recurring theme we ran into was that businesses already had RPA (pure-software scripts), and…
2025 · github.com · its alternatives →
- 19AA
2021 · autodevtech.com · its alternatives →
- 20
we had hundreds of discussions with engineering leaders over the past few months, and everyone's trying to understand where they are in the AI journey. we collected all this data into a benchmark and built a free grader to let you know where you stand. you answer on a 1–5 scale (e.g., autonomy runs from "suggestions only" to "agents own multi-hour workflows across code, infra, and external systems") - takes about 5 minutes. https://agent-benchmarks.com/software-factory/ waiting for your results!
Jul 2026 · agent-benchmarks.com · its alternatives →
- 21
Build better, faster Java code
2021 · cybench.io · its alternatives →
- 22ES
an exploration in reading and understanding code
2021 · artifacts.bypaulshen.com · its alternatives →
- 23
An open-source benchmark of how much code AI models add.
10d ago · parsimony.lasu.dev · its alternatives →
- 24MA
Run, compare, and share JavaScript benchmarks in your browser with BenchJS.
2024 · benchjs.com · its alternatives →
Also compare
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →