nowfound

Alternatives

Products that do what Mini-swe-agent achieves 65% on SWE-bench in 100 lines of python does

  1. 1SS
  2. 2PT
  3. 3AA
  4. 4AP

    2014 · github.com

  5. 5JA
  6. 6FB
  7. 7SA
  8. 8IW
  9. 9AA
  10. 10MP
  11. 11IC
  12. 12AM
  13. 13AR

    If you're interested in exploring what LLM-based agent systems these days actually do to solve certain benchmarks such as SWEBench or WebArena, we created a small leaderboard with our team, that allows to view a lot of public and OSS agent results including all the runtime traces (the step-by-step reasoning behind the scenes). Looking at traces is actually quite interesting, as they reveal a lot about the inner working and shortcomings of current agent system, e.g. see https://explorer.invariantlabs.ai/u/invariant/webarena--SteP... for an example trace.

    2024 · explorer.invariantlabs.ai

  14. 14LA
  15. 15AA
  16. 16RS

    What if your agent uses a different LM at every turn? We let mini-SWE-agent randomly switch between GPT-5 and Sonnet 4 and it scored higher on SWE-bench than with either model separately. GPT-5 by itself gets 65.0%, Sonnet 4 64.8%, but randomly switching at every step gets us 67.2% This result came pretty surprising to us. There's a few more experiments in the blog post.

    2025 · swebench.com

  17. 17OS

    May 2026 · 49agents.com

  18. 18MD

    We’re excited to share ML-Dev-Bench, a new open-source benchmark that tests AI agents on real-world ML development tasks. Unlike typical coding challenges or Kaggle-style competitions, our benchmark simulates end-to-end ML workflows including: - Dataset handling and preprocessing - Debugging model and code failures - Implementing new model architectures - Fine-tuning and improving existing models With 30 diverse tasks, ML-Dev-Bench evaluates agents across critical stages of ML development. To complement this, we built Calipers, a framework that provides systematic performance evaluation and…

    2025 · github.com

  19. 19FM

    2020 · github.com

  20. 20MA
  21. 21OA

    Nov 2025 · github.com

  22. 22ST
  23. 23MR
  24. 24LA

    We combined Stanford's ACE (agents learning from execution feedback) with the Reflective Language Model pattern. Instead of reading traces in a single pass, an LLM writes and runs Python in a sandbox to programmatically explore them - finding cross-trace patterns that single-pass analysis misses. The framework achieved 2x consistency improvement on τ2-bench.

    Mar 2026 · github.com

Ranked by how close each launch is in meaning, then by votes. Refine with a description →