A CTF challenge that let's you attack an LLM-based agent
Hello HN, given the recent interest in the security vulnerabilities of agentic LLM apps, we would love to hear everyone's thoughts on a Capture The Flag (CTF) challenge we built. The challenge is designed to test the robustness of LLM-based agent systems, showcasing how an attacker can extract sensitive information from the system. The challenge is part of the Invariant Summer '24 CTF series, where participants can play the challenge and compete for a prize pool of $1000. Let us know what you think and what attacks you tried. We are also open to any feedback on how we can improve with the…
A CTF challenge that let's you attack an LLM-based agent launched on August 22, 2024 with 5 votes, #2,614 of 5,471 launches that month and more than 50% of that year's launches.
What it does
In the maker’s words, at launch
Hello HN, given the recent interest in the security vulnerabilities of agentic LLM apps, we would love to hear everyone's thoughts on a Capture The Flag (CTF) challenge we built. The challenge is designed to test the robustness of LLM-based agent systems, showcasing how an attacker can extract sensitive information…
Does a similar job
all alternatives →
Agent ArenaJun 2026 · arena42.ai · ▲348Creator here. I built Agent Arena to answer a question that kept bugging me: when AI agents browse the web autonomously, how easily can they be manipulated by hidden instructions? How it works: 1. Send your AI agent to ref.jock.pl/modern-web (looks like a harmless web dev cheat sheet) 2. Ask it to summarize the page 3. Paste its response into the scorecard at wiz.jock.pl/experiments/agent-arena/ The page is loaded with 10 hidden prompt injection attacks -- HTML comments, white-on-white text, zero-width Unicode, data attributes, etc. Most agents fall for at least a few.…
FabraixMay 2026 · fabraix.com · ▲196We built Adversarial Cost to Exploit (ACE), a benchmark that measures the token expenditure an autonomous adversary must invest to breach an LLM agent. Instead of binary pass/fail, ACE quantifies adversarial effort in dollars, enabling game-theoretic analysis of when an attack is economically rational. We tested six budget-tier models (Gemini Flash-Lite, DeepSeek v3.2, Mistral Small 4, Grok 4.1 Fast, GPT-5.4 Nano, Claude Haiku 4.5) with identical agent configs and an autonomous red-teaming attacker. Haiku 4.5 was an order of magnitude harder to break than every other model; $10.21 mean…
PlaygroundJul 2026 · playground.fabraix.com · ▲198We build AI agents to find vulnerabilities in other AI agents. The playground started as an internal tool that we used to test our own offensive agent Nyx. But we realised that hacking an AI is about have as wide a distribution of attacks as possible, especially those which aren't in the distribution of the models. So we are launching playground as a platform to crowdsource agent jailbreaks. And although hacking AI is super fun and we are sure people will do it for free, we want to reward those who help us push forward the offensive frontier of AI. So we are committing $100K in prizes over…
We post-trained a model that pen tests instead of refusingJun 2026 · argusred.com · ▲93Anthropic and OpenAI's publicly available models are explicitly guard-railed so that they refuse offensive tasks. And their cyber-focussed models are gated for enterprises. This leaves SMEs and mid market open to major vulnerabilities. AI can be used as both an adversarial and defensive tool in the world of cyber. A worst case outcome is if only the adversaries have access. Meanwhile, most existing AI cyber tools are just wrappers. The problem is that they still have all the guardrails on from the foundation model where they will inherit its refusals. For this project we've post-trained a…
20+ Claude Code agents coordinating on real work (open source)Feb 2026 · github.com · ▲53Single-agent LLMs suck at long-running complex tasks. We’ve open-sourced a multi-agent orchestrator that we’ve been using to handle long-running LLM tasks. We found that single LLM agents tend to stall, loop, or generate non-compiling code, so we built a harness for agents to coordinate over shared context while work is in progress. How it works: 1. Orchestrator agent that manages task decomposition 2. Sub-agents for parallel work 3. Subscriptions to task state and progress 4. Real-time sharing of intermediate discoveries between agents We tested this on a Putnam-level math problem, but the…
Declaw Arena – a CTF-style challenge to break an AI agent in a microVMJul 2026 · declaw.ai · ▲8An interactive security challenge. Control an AI agent and try to exfiltrate data from a sandboxed environment protected by Declaw.
More ai this month
the category →
Paintings by AI models: programs run through a simulation of oil paint.
AI · 22d ago · stillwet.art
FastRouter.ai▲364Route requests to the right LLM for cost, latency & quality
AI · 23d ago · fastrouter.ai
Source: Show HN post ↗ · launch text quoted in part
Launched alongside, August 2024
the whole month →
Jupitrr AI▲1,195Jupitrr AI turns raw talking videos into scroll-stopping content for coaches and consultants. Research, script, record, edit, and publish in one app.
AI · 2024 · jupitrr.com
Hey there HN! We’re Joe and Stopa, and today we’re open sourcing InstantDB, a client-side database that makes it easy to build real-time and collaborative apps like Notion and Figma. Building modern apps these days involves a lot of schleps. For a basic CRUD app you need to spin up servers, wire up endpoints, integrate auth, add permissions, and then marshal data from the backend to the frontend and back again. If you want to deliver a buttery smooth user experience, you’ll need to add optimistic updates and rollbacks. We do these steps over and over for every feature we build, which can…
Dev tools · 2024 · github.com