Alternatives
Products that do what RedMirror: make any LLM find real bugs does
Make any LLM find real bugs in your code, and prove them
- 1
Works on 120b, 20b and even 8b models
27d ago · redmirror.io
- 2
- 3
- 4
- 5
- 6
- 7
Trace LLM requests + costs with OpenTelemetry monitoring
Oct 2025
- 8
- 9CB
I built a small benchmark to test CLI coding agents on blind bug detection. A challenger agent injects bugs and writes ground truth (`bugs.json`). A different reviewer agent audits the repo without seeing ground truth, and an LLM matcher scores bug-to-finding assignments. Current run: 50 repos, 150 challenges, 450 reviews, 2,603 injected bugs. Weighted detection: Claude 58.05%, Codex 37.84%, Gemini 27.81%. LLM-judge benchmarks are easy to get wrong, so I’d really appreciate critical feedback on benchmark fairness, scoring/matching methodology, and obvious failure modes I’m missing. Full…
Feb 2026 · github.com
- 10HW
Hello everyone! I’m thrilled to announce the latest feature from Mutahunter.ai, the ultimate tool for finding and fixing weaknesses in your code. We’ve designed Mutahunter to leverage mutation testing powered by advanced LLMs, helping you uncover vulnerabilities and enhance your code quality effortlessly. Introducing our newest feature: Detailed Mutation Testing Reports! After running our mutation tests, Mutahunter now generates comprehensive reports that clearly summarize: • Vulnerable code gaps • Test case gaps These reports significantly reduce the cognitive load on developers by…
2024 · github.com
- 11AB
Hey everyone, My friend and I built a simple bug fixing app that listens for alerts/issues from Sentry, contextualizes it against your codebase, and any other data sources you wish to connect (right now we support Notion, Google Docs, and Slack), and deploys an ai agent to write a PR for review in Github or Gitlab to solve the bug. Our current demo shows the end-to-end process for a trivial bug fix, but we have been testing it with open source python repos like http-pie, comparing how our agent solves a bug compared to a human engineer and it gets fairly close. We are working on adding…
2023 · resolvd.ai
- 12FB
2014 · bughunt.io
- 13IS
Hey HN! For that last 8 months I've been trying to make agents that can hack web applications to find vulnerabilities in them - An AI Security Tester. The system has 29 agents in total, a custom LLM Orchestration framework which works on the task-subtask architecture (old-school but works amazingly for my use case, and is pretty reliable) with custom agent calling mechanism. No Auo-Gen, Langchain and Crew AI - Everything custom built for pentesting. Each test runs in an isolated Kali linux environment (on AWS Fargate), where the agents have full access to the environment to undertake any…
2025
- 14CB
AI agents now have impressive reasoning capabilities. This raises an important question: how dangerous are these AI agents at identifying & exploiting web vulnerabilities? We created CVE-bench to find out (I'm one contributor of 16). To our knowledge CVE-bench is the first benchmark using real-world web vulnerabilities to evaluate AI agents' cyberattack capabilities. We included 40 CVEs from NIST's database, focusing on critical-severity vulnerability (CVSS > 9.0). To properly evaluate agents’ attacks, we built isolated environments with containerization and identified 8 common attack…
2025 · github.com
- 15GY
Hi all, I've been working on this devtool for 1 month now for myself at first and I'll be curious to see if it's something that could work for you as well. So basically, it detects bugs in your website in production from real user sessions, an llm clusters them by severity and it provides the complete context of the issue that you can copy-paste into your coding agent to fix it in one go. Why did I create it? I've been shipping fast with tools like Cursor and Claude Code. The problem? When bugs happen in production, these tools have zero context about what actually went wrong. Sentry is…
Nov 2025 · sonarly.dev
- 16AR
If you're interested in exploring what LLM-based agent systems these days actually do to solve certain benchmarks such as SWEBench or WebArena, we created a small leaderboard with our team, that allows to view a lot of public and OSS agent results including all the runtime traces (the step-by-step reasoning behind the scenes). Looking at traces is actually quite interesting, as they reveal a lot about the inner working and shortcomings of current agent system, e.g. see https://explorer.invariantlabs.ai/u/invariant/webarena--SteP... for an example trace.
2024 · explorer.invariantlabs.ai
- 17NL
Built this because I was tired of every AI tool shipping my data to someone else server n0x runs the full stack LLM inference via WebGPU, autonomous ReAct agents, RAG over your own docs, sandboxed Python execution via Pyodide all inside a single browser tab. No account No keys No backend Models download once, cache in IndexedDB permanently. Biggest challenge was context window budgeting for the agent loop and making the WASM vector search non-blocking. Happy to talk architecture. GitHub: https://github.com/ixchio/n0x | Live demo: https://n0x-three.vercel.app
Mar 2026 · n0xth.vercel.app
- 18OS
Hi HN, Matvey, Ildar, Joey, and Dominik here. If you're building LLM agents that use tools, you're probably worried about prompt injection attacks that can hijack those tools. We were too, and found that solutions like prompt-based filtering or secondary "guard" LLMs can be unreliable. Our thesis is that agent security should be handled at the network level between the agent and the LLM, just like a traditional web application firewall. So we built Archestra Platform: an open-source gateway that acts as a secure proxy for your AI agents. It's designed to be a deterministic firewall against…
Oct 2025 · archestra.ai
- 19
Coding agents craft arbitrary code so securing them is more complicated than red-teaming. We post trained a cyber-security small llm, changed how it reasons and supplemented our controls using program analysis techniques such as inline reference monitoring to outperform GPT5.5-xhigh on hard benchmarks like LinuxArena and SleightBench. Free product available at harden.run and full benchmarks in the blog post.
9d ago · harden.run
- 20AB
Hi HN, Zidan here. I’ve been experimenting with AI-assisted debugging and noticed a recurring gap: most tools optimize for agent-led exploration (ex: giving claude code a browser to click around and try to reproduce an issue). But in many cases, I've already found the bug myself. What I actually want is a way to hand the agent the exact context I just saw - without retyping steps, copying logs, or hoping it can reproduce the behavior. So we built FlowLens, an open-source MCP server + Chrome extension that captures browser context and lets coding agents inspect it as structured, queryable…
Nov 2025 · github.com
- 21RC
As a developer, one of the biggest obstacles I've encountered has been how to communicate with designers - especially, when it comes to constructing their designs. Over the years, I have come to realize that Designers and Developers just see the world differently - Developers see the world as defects in code text and Designers see the world as defects in pixels. This "language barrier" is what Redline (http://www.redline.cc) addresses -it allows Designers (or anyone really) to enter bugs visually and directly on the web page. Then, those mark-ups automatically creates a bug ticket in your…
2012
- 22IB
Logging and tools like Sentry are a thing of the past. A while back on a night out in northern Norway, I had to start debugging a critical production bug that broke the payment flow of my SaaS product. I had limited time to fix the bug or I would have lost about ~1K profit. Super stressful. I had logging and Sentry in place, but neither helped me reproduce or find the root cause of the bug. Ever since, I started thinking; why can’t we just have a tool that you setup once, and that allows us to reproduce every function call and function that the user ran before the bug? This is how the idea…
2023 · useflytrap.com
- 23OS
Hello HN, I’ve been building AI agents lately and ran into a common "Context Bloat" problem. When an agent has 20+ skills, stuffing every system prompt, reference doc, and tool definition into a single request quickly hits token limits and degrades model performance (the "lost in the middle" problem). To solve this, I built OpenSkills, an open-source SDK that implements a Progressive Disclosure Architecture for agent skills. The Core Concept: Instead of loading everything upfront, OpenSkills splits a skill into three layers: Layer 1 (Metadata): Light-weight tags and triggers (always loaded…
Jan 2026
- 247D
hi all. i’ve been shipping a small open project that tries to answer that question with evidence, not vibes. in 70 days it reached \~800 stars. the core claim is simple: many AI failures are not noise. they repeat because the geometry and ordering underneath are stable. if so, we should be able to name each failure mode, set acceptance targets, and stop shipping the same bug twice. ### what it is * a compact Problem Map of 16 reproducible failure modes in RAG and agents. * each item has a minimal fix and measurable gates. examples: * Semantic ≠ Embedding: metric and normalization mismatch.…
2025 · github.com
Ranked by how close each launch is in meaning, then by votes. Refine with a description →