A registry of agent benchmarks (including many OSS agent trajectories)
If you're interested in exploring what LLM-based agent systems these days actually do to solve certain benchmarks such as SWEBench or WebArena, we created a small leaderboard with our team, that allows to view a lot of public and OSS agent results including all the runtime traces (the step-by-step reasoning behind the scenes). Looking at traces is actually quite interesting, as they reveal a lot about the inner working and shortcomings of current agent system, e.g. see https://explorer.invariantlabs.ai/u/invariant/webarena--SteP... for an example trace.
A registry of agent benchmarks (including many OSS agent trajectories) launched on December 23, 2024 with 6 votes and more than 50% of that year's launches. The site was still online in September 2026, 1 year on.
What it does
A better way of testing, inspecting, and analyzing AI Agent traces. - invariantlabs-ai/explorer
Note: The hosted version of Invariant Explorer has been shut down in January 2026. If you're interested in Invariant Labs' continued work on AI security, check out Snyk's AI Security offering . # install Invariant package pip install invariant-ai # pull and launch Explorer application invariant explorer You can then access your Explorer instance via http://localhost . Data will be stored at ./data of the current working directory. To get started, run the following command. Make sure you first install Docker Compose . ./run.sh up To open the Explorer interface go to http://localhost . This will automatically launch a local Explorer instance on your machine. All data will be stored in ./data…from explorer.invariantlabs.ai
Does a similar job
all alternatives →
OSS Agent I built topped the TerminalBench on Gemini-3-flash-previewApr 2026 · github.com · ▲393Scored 65.2% vs google's official 47.8%, and the existing top closed source model Junie CLI's 64.3%. Since there are a lot of reports of deliberate cheating on TerminalBench 2.0 lately (https://debugml.github.io/cheating-agents/), I would like to also clarify a few things 1. Absolutely no {agents/skills}.md files were inserted at any point. No cheating mechanisms whatsoever 2. The cli agent was run in leaderboard compliant way (no modification of resources or timeouts) 3. The full terminal bench run was done using the fully open source version of the agent, no…
TrailsJan 2026 · trails-red.vercel.app · ▲81Hey all, Justin here. I previously built Phind, the AI search engine for developers. One of the biggest problems we had there was figuring out what went wrong with bad searches. We had tons of searches per day, but less than 1% of users gave any explicit feedback. So we were either manually digging through searches or making general system improvements and hoping they helped. This problem gets harder with agents. Traces are longer and more complex. It takes more effort to review them, so I'm building a tool that lets you analyze LLM outputs directly to help developers of LLM apps and agents…
Notte Browser Agents2025 · notte.cc · ▲275Hey HN, We’ve been building browser agents for a while. In production, we kept converging on the same pattern: deterministic scripts for the happy path, agents only for edge cases. So we built Demonstrate Mode. The idea is simple: You perform your workflow once in a remote browser. Notte records the interactions and generates deterministic automation code. How it works: - Record clicks, inputs, navigations in a cloud browser - Compile them into deterministic code (no LLM at runtime) - Run and deploy on managed browser infrastructure Closest analog is Playwright codegen but: - Infrastructure…
Coarena by CoastyAug 2026 · coarena.ai · ▲111We built a computer use agent once and got it benchmarked at the 2nd position in the biggest benchmark for this field but in the process, we realized one thing, how outdated and inefficient these benchmarks are and how they don't truly capture how these tasks went. So we thought a community led benchmarking arena would be the way to go about. https://coarena.ai/. You just have to put in a task and two side by side windows with hidden models will show up and do the same task and at the end you vote which one did better(or none). We have a leaderboard where we rank all the…
ReplicasJun 2026 · replicas.dev · ▲239Hey HN! I'm Connor, creator of Replicas. I'm a heavy Codex/Claude Code user, and I wanted to hand off tasks to agents like I would to an engineer - kick something off before work, let it scaffold while I'm at lunch, even come back to a ready PR. The problem: existing alternatives use workspace VMs that are black boxes. I have no access to any of them - they can't run tests, read logs, or use tools like the Playwright MCP (which I use to verify agents actually fixed what they claim to fix). Replicas gives agents real cloud workspaces where they can: - Run your test suite and read the…
Progress AI ObservabilityAug 2026 · telerik.com · ▲168Hi HN! We’re the team behind Fiddler, and we recently built an Agent Cache feature. It works by capturing an LLM request/response pair and, once caching is enabled, serving subsequent matching calls locally so duplicate requests never hit the provider. It’s meant to reduce three pains that show up together in agent development: token cost, feedback-loop latency, and output non-determinism. We’d love to hear your thoughts.
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗
