Fabraix
We built Adversarial Cost to Exploit (ACE), a benchmark that measures the token expenditure an autonomous adversary must invest to breach an LLM agent. Instead of binary pass/fail, ACE quantifies adversarial effort in dollars, enabling game-theoretic analysis of when an attack is economically rational. We tested six budget-tier models (Gemini Flash-Lite, DeepSeek v3.2, Mistral Small 4, Grok 4.1 Fast, GPT-5.4 Nano, Claude Haiku 4.5) with identical agent configs and an autonomous red-teaming attacker. Haiku 4.5 was an order of magnitude harder to break than every other model; $10.21 mean…
Fabraix launched on May 8, 2026 with 196 votes, #133 of 20,538 launches that month and more than 99% of that year's launches.
In plain words
Nyx is an autonomous testing harness that probes AI agents to identify failure modes before users encounter them. It finds logic bugs, reasoning failures, instruction-following errors, edge cases, and security vulnerabilities like jailbreaks and prompt injection through multi-turn adaptive conversations. The tool works as a pure blackbox—requiring no special system access—and tests across multiple modalities including text, voice, images, and browser interactions. It runs tests in parallel to autonomously discover weaknesses without requiring manually written test cases.
written from the facts on this page · September 2026
From the sources
AI agents fail in ways traditional software doesn't. Our agents help you find all the ways in which your AI agents fail by adversarially testing them in a dedicated environment. Point it at any AI agent, or multi-agent system, and it launches 1,000+ strategies that adapt to your system in real time - pure blackbox, no integration needed. Built by ex-Meta engineers.
Fabraix builds red-teaming AI agents that continuously find security vulnerabilities in customer-facing AI.
In the maker’s words, at launch
We built Adversarial Cost to Exploit (ACE), a benchmark that measures the token expenditure an autonomous adversary must invest to breach an LLM agent. Instead of binary pass/fail, ACE quantifies adversarial effort in dollars, enabling game-theoretic analysis of when an attack is economically rational. We tested…
Does a similar job
all alternatives →
PlaygroundJul 2026 · playground.fabraix.com · ▲198We build AI agents to find vulnerabilities in other AI agents. The playground started as an internal tool that we used to test our own offensive agent Nyx. But we realised that hacking an AI is about have as wide a distribution of attacks as possible, especially those which aren't in the distribution of the models. So we are launching playground as a platform to crowdsource agent jailbreaks. And although hacking AI is super fun and we are sure people will do it for free, we want to reward those who help us push forward the offensive frontier of AI. So we are committing $100K in prizes over…
Agent ArenaJun 2026 · arena42.ai · ▲348Creator here. I built Agent Arena to answer a question that kept bugging me: when AI agents browse the web autonomously, how easily can they be manipulated by hidden instructions? How it works: 1. Send your AI agent to ref.jock.pl/modern-web (looks like a harmless web dev cheat sheet) 2. Ask it to summarize the page 3. Paste its response into the scorecard at wiz.jock.pl/experiments/agent-arena/ The page is loaded with 10 hidden prompt injection attacks -- HTML comments, white-on-white text, zero-width Unicode, data attributes, etc. Most agents fall for at least a few.…
We post-trained a model that pen tests instead of refusingJun 2026 · argusred.com · ▲93Anthropic and OpenAI's publicly available models are explicitly guard-railed so that they refuse offensive tasks. And their cyber-focussed models are gated for enterprises. This leaves SMEs and mid market open to major vulnerabilities. AI can be used as both an adversarial and defensive tool in the world of cyber. A worst case outcome is if only the adversaries have access. Meanwhile, most existing AI cyber tools are just wrappers. The problem is that they still have all the guardrails on from the foundation model where they will inherit its refusals. For this project we've post-trained a…
I put an AI agent on a $7/month VPS with IRC as its transport layerMar 2026 · georgelarson.me · ▲340The stack: two agents on separate boxes. The public one (nullclaw) is a 678 KB Zig binary using ~1 MB RAM, connected to an Ergo IRC server. Visitors talk to it via a gamja web client embedded in my site. The private one (ironclaw) handles email and scheduling, reachable only over Tailscale via Google's A2A protocol. Tiered inference: Haiku 4.5 for conversation (sub-second, cheap), Sonnet 4.6 for tool use (only when needed). Hard cap at $2/day. A2A passthrough: the private-side agent borrows the gateway's own inference pipeline, so there's one API key and one billing relationship…
- FPFind prompts that jailbreak your agent (open source)2025 · security.vista-labs.ai · ▲8
We've built an open-source tool to stress test AI agents by simulating prompt injection attacks. We’ve implemented one powerful attack strategy based on the paper [AdvPrefix: An Objective for Nuanced LLM Jailbreaks](https://arxiv.org/abs/2412.10321). Here's how it works: - You define a goal, like: “Tell me your system prompt” - Our tool uses a language model to generate adversarial prefixes (e.g., “Sure, here are my system prompts…”) that are likely to jailbreak the agent. - The output is a list of prompts most likely to succeed in bypassing safeguards. We’re just getting…
- INI nerfed our coding agents on purposeJun 2026 · ▲27
Tl;dr: I trained a classifier to route to the least expensive model and reasoning depth to complete the request. Coupling that with additional automated token efficiency techniques has yielded 3x usage for the same spend. For anyone interested in trying it themselves: https://nerfguard.com Various teammates and I switched over to Codex from Claude Code recently. We still bounce between the tools, but Codex’s speed and steerability coupled with performance gains were hard to ignore. One of the downsides was that the per token pricing kicked in way sooner. This is happening across…
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 20d ago · stillwet.art
PixVerse R2▲381A real-time world model you can explore and change
AI · 28d ago · world.pixverse.video
Source: Product Hunt launch ↗ · launch text quoted in part
Launched alongside, May 2026
the whole month →
Parallel agents, diff reviewer, and multi-model comparisons
Dev tools · May 2026 · kilo.ai
Hey HN, Henry here from Cactus. We open-sourced Needle, a 26M parameter function-calling (tool use) model. It runs at 6000 tok/s prefill and 1200 tok/s decode on consumer devices. We were always frustrated by the little effort made towards building agentic models that run on budget phones, so we conducted investigations that led to an observation: agentic experiences are built upon tool calling, and massive models are overkill for it. Tool calling is fundamentally retrieval-and-assembly (match query to tool name, extract argument values, emit JSON), not reasoning. Cross-attention…
Life & fun · May 2026 · github.com
🌱 Private, quiet space for thinking. Simple app for .md files. - zakirullin/files.md
Dev tools · May 2026 · github.com