AI · alternatives · 2026
24 alternatives to Extreme Mutation Testing to Optimize LLM Agent-Based Mutation Testing
Hey HN, Steven here from CodeIntegrity https://github.com/codeintegrity-ai/mutahunter We’re obsessed with automating software testing, specifically mutation testing, and have been frustrated with its slow adoption despite its proven success. I have a pretty different perspective on mutation testing and have shared my thoughts on its current state - https://www.jungsteven.com/blog/2024-07-03-past-present-future-mutation-testing Over the past few months, we’ve developed a new mutation testing tool that’s easy to use and compatible with any programming…
Below are 24 products that do a similar job, ranked by how close each is in meaning and then by launch-day votes. Extreme Mutation Testing to Optimize LLM Agent-Based Mutation Testing launched in 2024; newer entries below may have overtaken it.
- 1
Hello everyone! I’m thrilled to announce the latest feature from Mutahunter.ai, the ultimate tool for finding and fixing weaknesses in your code. We’ve designed Mutahunter to leverage mutation testing powered by advanced LLMs, helping you uncover vulnerabilities and enhance your code quality effortlessly. Introducing our newest feature: Detailed Mutation Testing Reports! After running our mutation tests, Mutahunter now generates comprehensive reports that clearly summarize: • Vulnerable code gaps • Test case gaps These reports significantly reduce the cognitive load on developers by…
2024 · github.com · its alternatives →
- 2
Background: We were inspired by how aider.chat was making use of pageranked AST and realized that we can use this to power high quality mutation testing using LLMs. Check it out!
2024 · github.com · its alternatives →
- 3
QA.tech 1.0▲413Hi, I'm Daniel, CEO of QA.tech. We've spent the past 18 months developing a cool tool that allows you to run end-to-end tests on any web application using natural language. Check it out and let us know what you think! I used to be a CTO at a B2C marketplace and was taken aback that all qa agencies we talked to wanted to create a custom suite of tests from scratch. Even though we were essentially a e-com with the same functionality as everyone else. Why weren't there any best practices? Some time later my co-founders and I had moved on and were looking for the next thing to do. AI agents…
2025 · qa.tech · its alternatives →
- 4
TestSprite 1.0▲753First AI agent automating entire software testing process
2025 · testsprite.com · its alternatives →
- 5
I spent a few hours last weekend testing whether AI can replace code by executing directly. Built a contact manager where every HTTP request goes to an LLM with three tools: database (SQLite), webResponse (HTML/JSON/JS), and updateMemory (feedback). No routes, no controllers, no business logic. The AI designs schemas on first request, generates UIs from paths alone, and evolves based on natural language feedback. It works—forms submit, data persists, APIs return JSON—but it's catastrophically slow (30-60s per request), absurdly expensive ($0.05/request), and has zero UI…
Nov 2025 · github.com · its alternatives →
- 6MF
2024 · mutahunter.ai · its alternatives →
- 7
Agent Simulate▲413Test your AI apps with thousands of digital humans
2025 · agentsimulate.com · its alternatives →
- 8
Trails▲81Hey all, Justin here. I previously built Phind, the AI search engine for developers. One of the biggest problems we had there was figuring out what went wrong with bad searches. We had tons of searches per day, but less than 1% of users gave any explicit feedback. So we were either manually digging through searches or making general system improvements and hoping they helped. This problem gets harder with agents. Traces are longer and more complex. It takes more effort to review them, so I'm building a tool that lets you analyze LLM outputs directly to help developers of LLM apps and agents…
Jan 2026 · trails-red.vercel.app · its alternatives →
- 9
Recently I've been working on making LLM evaluations fast by using bayesian optimization to select a sensible subset. Bayesian optimization is used because it’s good for exploration / exploitation of expensive black box (paraphrase, LLM). I would love to hear your thoughts and suggestions on this!
2024 · github.com · its alternatives →
- 10
I have been working in AI space for a while now, first at FAANG with ML since 2021, then with LLM in start-ups since early 2023. I think LLM Application development is extremely iterative, more so than any other types of development. This is because to improve an LLM application performance (accuracy, hallucinations, latency, cost), you need to try various combinations of LLM models, prompt templates (e.g., few-shot, chain-of-thought), prompt context with different RAG architecture, different agent architecture, and more. There are thousands of possible combinations and you need a process…
2024 · github.com · its alternatives →
- 11
- 12OS
Everyone saw the AlphaEvolve hype. I got obsessed with how it might work under the hood and decided to just build it myself. My setup uses GPT-4.1 to mutate matrix multiplication code, guided by a bunch of hand-crafted mutation strategies (loop reordering, tiling, Strassen, etc.). Each candidate is evaluated on both speed and accuracy. Then I apply Pareto selection with crowding distance to evolve better ones over generations. I ran into all the usual LLM reward hacks-returning the input, calling np.dot, etc. So I forced primitive-only implementations and tightly constrained the mutation…
2025 · its alternatives →
- 13
Hi HN! I’m a founder at Nextmv (YC 20) [1] We’ve been building out optimization algorithm testing capabilities (acceptance tests, etc.) and just shipped our first pass at shadow testing [2, 3]. In our experience, tools like shadow testing save time and build confidence in decision models, but tools also take time to build and maintain. We’ve seen shadow testing tools in the machine learning and MLOps space [4], but not so much in the operations research community. A lot of folks here [5] seem experienced with optimization models and we’d love to have your feedback! What do you like? What…
2023 · nextmv.io · its alternatives →
- 14
Spelltest framework simulates conversations between AI ‘synthetic users' in an environment to test and refine LLM-based applications. It ensures your app converse with utmost accuracy and relevance. Post-chat, Spelltest assesses responses, providing qualitative and quantitative feedback on performance. Suitable for both chat and completion modes. When to use: - After modifying your prompt. - When your LLM provider updates. - As a CI step for you repo. All feedback and collaborations appreciated!
2023 · github.com · its alternatives →
- 15
I wanted to test mobile apps in plain English instead of relying on brittle selectors like XPath or accessibility IDs. With a vision-based agent, that part actually works well. It can look at the screen, understand intent, and perform actions across Android and iOS. The bigger problem showed up around how tests are defined and maintained. When test flows are kept outside the codebase (written manually or generated from PRDs), they quickly go out of sync with the app. Keeping them updated becomes a lot of effort, and they lose reliability over time. I then tried generating tests directly from…
Apr 2026 · github.com · its alternatives →
- 16WH
Hey guys, I'm investing a ton of time on writing unit tests, for both enterprise and personal projects. I came up with the idea to make extension for AI-generated tests and cases within a VS Code. Happy to hear feedback, both positive and negative.
2024 · marketplace.visualstudio.com · its alternatives →
- 17ET
Hey Hacker News, For the last 2 months, I've been working on a testing agent to free developers from the endless maintenance of end-to-end tests. You just push up a PR, and our agent analyzes the code changes and automatically visits the preview to test things out like a real human! We also support describing tests in English (or even in the PR description), and we'll go through your site whenever you want via a GitHub action to test and make sure various core flows continue to work as expected. We are looking for early testers and are giving out a generous free tier! Just sign up on the…
2025 · playmatic.ai · its alternatives →
- 18KA
I built this because Cursor, Claude Code and other agentic AI tools kept giving me tests that looked fine but failed when I ran them. Or worse - I'd ask the agent to run them and it would start looping: fix tests, those fail, then it starts "fixing" my code so tests pass, or just deletes assertions so they "pass". Out of that frustration I built KeelTest - a VS Code extension that generates pytest tests and executes them, got hooked and decided to push this project forward... When tests fail, it tries to figure out why: - Generation error: Attemps to fix it automatically, then tries again -…
Jan 2026 · keelcode.dev · its alternatives →
- 19
Hi HN, I built this open-source LLM red teaming tool based on my experience scaling LLMs at a big co to millions of users... and seeing all the bad things people did. How it works: - Uses an unaligned model to create toxic inputs - Runs these inputs through your app using different techniques: raw, prompt injection, and a chain-of-thought jailbreak that tries to re-frame the request to trick the LLM. - Probes a bunch of other failure cases (e.g. will your customer support bot recommend a competitor? Does it think it can process a refund when it can't? Will it leak your user's address?) -…
2024 · promptfoo.dev · its alternatives →
- 20
Spec27▲34Hi HN! We’re a team of ML validation specialists and we’ve been building /Spec27, a tool for testing whether AI agents still do their job safely and reliably as models, prompts, tools, and surrounding systems change. We started working on this because a lot of current LLM evaluation work seems aimed at scoring general model behavior, while many teams are deploying systems that have a specific mission to fulfill. Many of the tools also assume you have full access to the agent stack and traces so you can place SDKs and Gateways, but a lot of agents are being created on vendor platforms…
Apr 2026 · spec27.ai · its alternatives →
- 21IN
Tl;dr: I trained a classifier to route to the least expensive model and reasoning depth to complete the request. Coupling that with additional automated token efficiency techniques has yielded 3x usage for the same spend. For anyone interested in trying it themselves: https://nerfguard.com Various teammates and I switched over to Codex from Claude Code recently. We still bounce between the tools, but Codex’s speed and steerability coupled with performance gains were hard to ignore. One of the downsides was that the per token pricing kicked in way sooner. This is happening across…
Jun 2026 · its alternatives →
- 22
Hi Hacker News! We're launching Zalor, an agent testing platform. Agents often break when you tweak system prompts, swap models, or add tools. Zalor automatically generates test scenarios and evaluates your agent so you know it's reliable before deploying to production. We currently support the OpenAI Agents SDK and are onboarding other frameworks. A GitHub integration is coming so you can get feedback on every update. Looking forward to hearing feedback from people building agents.
Mar 2026 · agents.zalor.ai · its alternatives →
- 23
At testup.io we have been working for a while to bring artificial intelligence to the field of test automation. Just a few years ago, the primary challenge laid in accurately identifying UI elements following minor structural changes, such as updates to IDs or paths. The emergence of Large Language Models (LLMs) raised the bar for what it meant to be smart. Now, we anticipate the robot to do lots of things autonomously, such as retry in cases of unresponsiveness or handle minor error reports. A more challenging, but soon expected feature, would involve the test robot navigating your web shop…
2024 · github.com · its alternatives →
- 24ME
We are building a VM that helps you simulate realistic production conditions, model latencies, different interleaving, user requests, and find bugs. Every non-deterministic property is turned into a knob you or a coding agent can control. We have helped teams perfectly reproduce support incidents and found bugs in some of the world's most well tested software (including a database).
Jun 2026 · workers.io · its alternatives →
Also compare
- How we use LLMs to find testing gaps, vulnerabilities in codebases alternatives
- Mutahunter – LLMs to support mutating testing for all major languages alternatives
- QA.tech 1.0 alternatives
- TestSprite 1.0 alternatives
- Why write code if the LLM can just do the thing? (web app experiment) alternatives
- Mutahunter – Fast, language agnostic, software testing using LLM agents alternatives
Ranked by how close each launch is in meaning, then by votes. Prices were read from each product’s own site when checked and can change. Refine with your own description →