Alternatives
Products that do what jj-benchmark – Evaluating AI agents on Jujutsu version control does
Hi HN, Meng from TabbyML here. We decided to build this simply because we find Jujutsu (jj) really interesting, and many folks on our team have started trying it out recently. Since it introduces a very different workflow compared to traditional Git, we thought it would be a fun challenge to see how well current AI coding agents can actually use it. To build this, we created a semi-automated pipeline. We used AI to research the official Jujutsu documentation and websites, which then helped us bootstrap a dataset of 63 distinct evaluation tasks. Each task includes instructions, bootstrap…
- 1
- 2LA
Please meet Loadjitsu, my weekend project, years in the making. Over the years while building different apps and sites, I always felt that I need a modern load testing software.Tools like JMeter, ab are not very easy to use and it seems innovation in load testing which is a crucial part of any software release cycle has been ignored. This is my third attempt at making Loadjitsu, I am so glad that I can finaly release this. A bit more about the software 1. Powered by golang you can run load tests for tens of thousands of connections per second on very average hardware. 2. Cross platform, run…
2022 · loadjitsu.com
- 3AR
If you're interested in exploring what LLM-based agent systems these days actually do to solve certain benchmarks such as SWEBench or WebArena, we created a small leaderboard with our team, that allows to view a lot of public and OSS agent results including all the runtime traces (the step-by-step reasoning behind the scenes). Looking at traces is actually quite interesting, as they reveal a lot about the inner working and shortcomings of current agent system, e.g. see https://explorer.invariantlabs.ai/u/invariant/webarena--SteP... for an example trace.
2024 · explorer.invariantlabs.ai
- 4LO
Hi HN, Martin, Nils, and Jannes here. We are building Legit, an open source version control and collaboration layer for AI agents and AI native applications. You can find the repo here https://github.com/Legit-Control/monorepo and the website here https://legitcontrol.com Over the last years, we worked on multiple developer tools and AI driven products. As soon as we started letting agents modify real files and business critical data, one problem kept showing up. We could not reliably answer what changed, why it changed, or how to safely undo it. Today, most AI…
Jan 2026
- 5SR
Hello all, I'm a software developer. Over the last few months more and more of my work has turned into using coding agents instead of typing the whole code myself. Usually a few claude sessions at once, sometimes codex, one per feature or per revealed bug. I ran them in a split terminal for a few weeks, and quickly spotted two main problems. The first is that I couldn't easily tell which agent was stuck waiting on me and which was still working, so I'd cycle through sessions and checking on them. The second one: agents sharing a single branch step on each other. Two of them could be editing…
Jul 2026 · shikigami.dev
- 6MD
We’re excited to share ML-Dev-Bench, a new open-source benchmark that tests AI agents on real-world ML development tasks. Unlike typical coding challenges or Kaggle-style competitions, our benchmark simulates end-to-end ML workflows including: - Dataset handling and preprocessing - Debugging model and code failures - Implementing new model architectures - Fine-tuning and improving existing models With 30 diverse tasks, ML-Dev-Bench evaluates agents across critical stages of ML development. To complement this, we built Calipers, a framework that provides systematic performance evaluation and…
2025 · github.com
- 7YA
Hey HN — Been deep in vibe coding with AI tools like Cursor and Windsurf — but sometimes, AI edits don’t go as planned. You hit revert… and only some files roll back, or your repo ends up in a weird state. Especially messy when you’ve got multiple chats open and you're mid-flow. So I built YOYO — a lightweight version control layer for the fast, chaotic phase of coding where you're exploring, iterating, and letting AI throw stuff at your repo. It works across Cursor, Windsurf, and VSCode — wherever you’re coding with AI right now. Not trying to replace Git. Git’s great when you’re ready to…
2025 · runyoyo.com
- 8RA
Hi HN! Sean from MindStudio here. I wanted to share something we've been working on that I think introduces some new ideas into the "AI coding agent" space. Remy is an AI agent that builds full-stack TypeScript apps from a spec written in a new flavor of annotated markdown. The spec has two layers: prose describing what the app does, and annotations that carry the technical precision (data types, edge cases, validation rules, code snippets). The agent then "compiles" this into code: backend methods, typed schemas, frontends, test scenarios, and everything else are derived artifacts of the…
Apr 2026 · remy.msagent.ai
- 9BA
I built CodeLens.AI - a tool that compares how 6 top LLMs (GPT-5, Claude Opus 4.1, Claude Sonnet 4.5, Grok 4, Gemini 2.5 Pro, o3) handle your actual code tasks. How it works: - Upload code + describe task (refactoring, security review, architecture, etc.) - All 6 models run in parallel (~2-5 min) - See side-by-side comparison with AI judge scores - Community votes on winners (blind voting) - Each evaluation gets reflected in the overall AI model leaderboard, showing us best ones Why I built this: Existing benchmarks (HumanEval, SWE-Bench) don't reflect real-world developer tasks. I wanted to…
Oct 2025 · codelens.ai
- 10YS
We been helping mid-market companies for the past 1.5 years and finally ready to share the internal platform publicly. Yansu (严肃) is a AI coding platform that use spec + TDD to build complex software projects. It is more like a SOP than coding agent. We focus on understanding requirements and checking outcomes against those requirements while iterating the code based on the tests. Yansu tries to learn as much tribal knowledge as possible. These are things you don’t write down in google doc or Notion. Yansu absorbs these knowledge by continuously talking to users and distilling learning from…
Nov 2025 · twitter.com
- 11JA
A few months ago I stumbled on obra's superpowers repository https://github.com/obra/superpowers. I really liked the approach and idea that you enforce discipline for your agent through a skill-based workflow. Even though coding agents (copilot included) have become a lot better at natively handling complex tasks, they still wander off and lose track of things. I really liked how superpowers fixed this and how it enabled long-running sessions without the agent losing its "focus". So I decided to build a Copilot tailored skill suite around the core idea of superpowers. I…
May 2026 · github.com
- 12BY
Hi HN. We launched a free AI Coding Risk Assessment tool to help engineering teams and businesses benchmark the security and compliance posture of their AI coding workflows and policies against peers in the industry. This anonymous 24-question survey delivers: - A 0–100 risk score that measures your AI coding security posture - A live benchmark that compares your AI-assisted development practices with peers - A research-based checklist that identifies improvement areas We're seeing more and more clients signal their concerns about the sudden increase of source code written by AI coding…
Nov 2025
- 138B
Hey all, Justin here. I previously built Phind, the AI search engine for developers. One of the biggest problems we had there was figuring out what went wrong with bad searches. We had tons of searches per day, but less than 1% of users gave any explicit feedback. So we were either manually digging through searches or making general system improvements and hoping they helped. This problem gets harder with agents. Traces are longer and more complex. It takes more effort to review them, so I'm building a tool that lets you analyze LLM outputs directly to help developers of LLM apps and agents…
Jan 2026 · trails-red.vercel.app
- 14IB
I’ve spent the last 2.5 months building a product that runs LLM-powered code reviews on my pull requests — and I just launched it. The tool is built specifically for solo developers. You install it on your repo, trigger a scan by creating a pull request, and it leaves structured review comments using OpenAI under the hood. Funnily enough, I used the dev version of this app to review its own pull requests while building it. It helped me spot bugs, simplify structure, and keep quality high — all with minimal need for another human in the loop. Things I want to try out in the next months : -…
2025 · codii.dev
- 15AD
Hi all, I threw together a small prototype I am calling “Notepad.ai”. A new take on UIs for interacting with LLMs. While I enjoy using LLM’s in the chat format I wanted to see what it would be like to do it in a more long form style. It let’s you write in a pretty free form, much like Window’s Notepad, but you can choose to hit ctrl+[ to analyze the text with a preset prompt of your choosing. It has a few other small features. It’s WIP and very experimental. I would appreciate any feedback or thoughts. Video: https://youtu.be/ntdlgFmSxQY Live Demo:…
2024 · github.com
- 16WB
Hey HN, After GPT-3 created waves in the tech industry, a lot of AI tools were emerging and with that, some AI website builders But the results seemed way too generic to us. It felt like the developers were rushing to catch the wave instead of building a proper tool We took our time, did months of RnD and finally came up with something better than what others in the market are doing. It’s got better design output. While it’s still in beta, I wanted to show HN what we did. Will appreciate the feedback when you guys try it out. Here is the link to signup for the beta:…
2024 · dorik.com
- 17SO
hello everyone, my first post! AA here, founder of ⌘ Langbase.com — we are a developer platform for building and scaling serverless AI memory agents. I know surveys can be boring, but this one’s different—it’s interactive! That's very much intentional. My team and I have been up for the last 21 hours putting together this report. This was a looot of work, so I hope y'all like it. Introducing … State of AI Agents 2024 report On Langbase, we processed 184 billion tokens and handled 786 million AI agent runs from 36K developers. From all that data plus insights from 3.4K builders who filled out…
2024 · langbase.com
- 18BC
We are a small group of undergrads interested in building human in the loop coding agents. We dream of a world where building complex agent workflows feels as simple and creative as playing with legos. When we were building stuff we needed a tool that made it easy to try out different code embedding models so that we could see which ones worked best in different scenarios and understand their strengths and weaknesses. So to speed that process up we made PurpleSearch an 'instant' search engine for your local codebases. This tool lets you quickly deploy any open source embedding model on…
2025
- 19CB
AI agents now have impressive reasoning capabilities. This raises an important question: how dangerous are these AI agents at identifying & exploiting web vulnerabilities? We created CVE-bench to find out (I'm one contributor of 16). To our knowledge CVE-bench is the first benchmark using real-world web vulnerabilities to evaluate AI agents' cyberattack capabilities. We included 40 CVEs from NIST's database, focusing on critical-severity vulnerability (CVSS > 9.0). To properly evaluate agents’ attacks, we built isolated environments with containerization and identified 8 common attack…
2025 · github.com
- 20MA
This weekend I built a multi-agent coding system which, quite unexpectedly, beat Claude Code on Stanford's Terminal Bench! The architecture is straightforward, consisting of an orchestrator agent that deploys explorer & coder subagents to complete complex terminal based tasks, utilising an intelligent context sharing mechanism along the way which makes it all work. The repo has a lot of technical details, and all the code and prompts for you to play around with if you'd like! I had a lot of fun making this, I hope you have fun reading the README, using it yourself, or even extending it! As…
2025 · github.com
- 21OS
Hey HN! I'm one of the cofounders of Sourcebot, an open source alternative to Sourcegraph. Sourcebot lets you index thousands of repos across multiple platforms (GitHub, GitLab, Bitbucket), and gives you a powerful interface to search across them. You can learn more in our original HN launch post: https://news.ycombinator.com/item?id=41711032 We just added an AI code review agent that reviews your PRs and automatically detects issues that a human reviewer may have missed. We've been using an AI code review agent for a few weeks now, and it regularly catches issues that we…
2025 · docs.sourcebot.dev
- 22NT
Hi HN, Tanooj and Aziz here from Nerve. We’re building a work AI that handles full end to end workflows rather than just chat replies. It starts with proactively figuring out what’s important to you - anything you need to take action on or updates on any projects - and then moves all the way through the process. Nerve gathers the relevant information, drafts documents, writes jira tickets, sends emails, etc. What this looks like in practice is: say an account executive has a sales call that gets recorded into Gong, Nerve automatically picks up the call, and extracts the next steps. Users can…
Dec 2025 · usenerve.com
- 23AB
Hi HN, Zidan here. I’ve been experimenting with AI-assisted debugging and noticed a recurring gap: most tools optimize for agent-led exploration (ex: giving claude code a browser to click around and try to reproduce an issue). But in many cases, I've already found the bug myself. What I actually want is a way to hand the agent the exact context I just saw - without retyping steps, copying logs, or hoping it can reproduce the behavior. So we built FlowLens, an open-source MCP server + Chrome extension that captures browser context and lets coding agents inspect it as structured, queryable…
Nov 2025 · github.com
- 24TS
Code review is the new bottleneck. "Tests passing" is no longer sufficient to trust the changes, and the (human) cost of evaluating the quality and robustness of new agent-written contributions is skyrocketing. We built Topos to evaluate code quality based on the structural properties of the programs themselves. We map your files to graphs (AST, CFG, CPG, MDG) and calculate metrics that can characterize how simple, composable, or secure your programs are. Agents can use this tool as they write and optimize based on your preferences. And yes, the inspiration for the repository is from…
Jun 2026 · krv.ai
Ranked by how close each launch is in meaning, then by votes. Refine with a description →