A CTF challenge that let's you attack an LLM-based agent
Hello HN, given the recent interest in the security vulnerabilities of agentic LLM apps, we would love to hear everyone's thoughts on a Capture The Flag (CTF) challenge we built. The challenge is designed to test the robustness of LLM-based agent systems, showcasing how an attacker can extract sensitive information from the system. The challenge is part of the Invariant Summer '24 CTF series, where participants can play the challenge and compete for a prize pool of $1000. Let us know what you think and what attacks you tried. We are also open to any feedback on how we can improve with the…
What it does
In the maker’s words, at launch
Hello HN, given the recent interest in the security vulnerabilities of agentic LLM apps, we would love to hear everyone's thoughts on a Capture The Flag (CTF) challenge we built. The challenge is designed to test the robustness of LLM-based agent systems, showcasing how an attacker can extract sensitive information from the system. The challenge is part of the Invariant Summer '24 CTF series, where participants can play the challenge and compete for a prize pool of $1000. Let us know what you think and what attacks you tried. We are also open to any feedback on how we can improve with the next challenges. CTF Details: https://invariantlabs.ai/ctf-challenge-24 Discord: https://discord.gg/dZuZfhKnJ4 Play the CTF: https://invariantlabs.ai/play-ctf-challenge-24
Does the same job
all alternatives →- RURewardHackBench: Using sandboxes to stop agents from cheatingJun 2026 · github.com · ▲9
hey all, happy to share research i've been working on for islo.dev in recent months. ever since the cheating agents (https://debugml.github.io/cheating-agents/) paper came out, revealing reward hacking was 4x more prevalent than previously estimated, i've been looking into how we can deal with the issue the common approach (taken by the tbench team) is post hoc trajectory analysis. i've been interested in the idea of reframing the problem as an endpoint security problem and tackling it via sandbox i hope you find it interesting, and thanks to the islo.dev team for…
- ISI spent 8 months trying to make LLMs Hack2025 · ▲6
Hey HN! For that last 8 months I've been trying to make agents that can hack web applications to find vulnerabilities in them - An AI Security Tester. The system has 29 agents in total, a custom LLM Orchestration framework which works on the task-subtask architecture (old-school but works amazingly for my use case, and is pretty reliable) with custom agent calling mechanism. No Auo-Gen, Langchain and Crew AI - Everything custom built for pentesting. Each test runs in an isolated Kali linux environment (on AWS Fargate), where the agents have full access to the environment to undertake any…
- ARA registry of agent benchmarks (including many OSS agent trajectories)2024 · explorer.invariantlabs.ai · ▲6
If you're interested in exploring what LLM-based agent systems these days actually do to solve certain benchmarks such as SWEBench or WebArena, we created a small leaderboard with our team, that allows to view a lot of public and OSS agent results including all the runtime traces (the step-by-step reasoning behind the scenes). Looking at traces is actually quite interesting, as they reveal a lot about the inner working and shortcomings of current agent system, e.g. see https://explorer.invariantlabs.ai/u/invariant/webarena--SteP... for an example trace.
- OSOpenSkills – Stop bloating your LLM context with unused instructionsJan 2026 · ▲9
Hello HN, I’ve been building AI agents lately and ran into a common "Context Bloat" problem. When an agent has 20+ skills, stuffing every system prompt, reference doc, and tool definition into a single request quickly hits token limits and degrades model performance (the "lost in the middle" problem). To solve this, I built OpenSkills, an open-source SDK that implements a Progressive Disclosure Architecture for agent skills. The Core Concept: Instead of loading everything upfront, OpenSkills splits a skill into three layers: Layer 1 (Metadata): Light-weight tags and triggers (always loaded…
- WMWe made a game this week-end!2014 · ▲9
Hi Hacker News! My friend and I [1] would like to share with you something we did this week-end. We really like the idea of game jams like the Ludum Dare, and we’d like to participate, but it’s often complicated for us, as we are often limited by the rules inherent to this kind of competition: we have to be available at specific dates, and the rules doesn’t always work for us. For example: if you want to participate as a team in a Ludum Dare compo, you have 72h instead of 48h, and you are not required to publish your code. We realized that what we really like is: - To work together. - The…
- TATeapot – A methodology for pen testing voice AI agentsFeb 2026 · redcaller.com · ▲7
Hello HN, I am Brian Cardinale, a penetration tester and security researcher at SecureCoders. We have been performing more and more AI based security assessments. We were presented a unique challenge of testing a system where the only interface was voice based, and as much as I like talking on the phone , we decided to create a test harness to facilitate the actual testing in a more systematic way. The technical test harness was the easy part, though. Creating test goals and attack strategies to help facilitate repeated and comprehensive testing became the real challenge. As such, we have…
More ai this month
the category →
I trained a 125M-parameter transformer to autocomplete piano performances in real time (~108 notes/sec on an iPhone 15). The idea is basically GitHub Copilot or Tabnine, except instead of prompting it with code, you prompt it by playing a few notes on a MIDI piano. The model then continues what you played, entirely on-device. The app is free if anyone wants to try it. Happy to answer questions about the model, training, Core ML, or the many things that didn't work.
AI · 17d ago · simedw.com
Astute▲585Automate your B2B brand going viral, with new media creators
AI · 18d ago · company-app.joinastute.com
Hey HN, Henry from Cactus here! We previously released Cactus Needle, a 14MB agentic LLM for tool call, device use, and structured extraction for phones, wearables, smart homes, small robots and microcontrollers. We got really great feedback here, and have now incorporated the suggestions to release Needle 2. The whole model is a single 14MB binary that runs a full session in 28MB of RAM; 45m parameters at 2bit compression. Needle hits 500 tokens/sec decode speed on a Raspberry Pi 5, sits between 400-1,500 tokens/sec on VR devices like Meta Quest 3S and Apple Vision Pro, and ranges…
AI · 26d ago · cactuscompute.com
Launched alongside, August 2024
the whole month →
- IY
Life & fun · 2024 · ytch.xyz
- IA
Hey there HN! We’re Joe and Stopa, and today we’re open sourcing InstantDB, a client-side database that makes it easy to build real-time and collaborative apps like Notion and Figma. Building modern apps these days involves a lot of schleps. For a basic CRUD app you need to spin up servers, wire up endpoints, integrate auth, add permissions, and then marshal data from the backend to the frontend and back again. If you want to deliver a buttery smooth user experience, you’ll need to add optimistic updates and rollbacks. We do these steps over and over for every feature we build, which can…
Dev tools · 2024 · github.com