Alternatives
Products that do what RewardHackBench: Using sandboxes to stop agents from cheating does
hey all, happy to share research i've been working on for islo.dev in recent months. ever since the cheating agents (https://debugml.github.io/cheating-agents/) paper came out, revealing reward hacking was 4x more prevalent than previously estimated, i've been looking into how we can deal with the issue the common approach (taken by the tbench team) is post hoc trajectory analysis. i've been interested in the idea of reframing the problem as an endpoint security problem and tackling it via sandbox i hope you find it interesting, and thanks to the islo.dev team for…
- 1OA
Scored 65.2% vs google's official 47.8%, and the existing top closed source model Junie CLI's 64.3%. Since there are a lot of reports of deliberate cheating on TerminalBench 2.0 lately (https://debugml.github.io/cheating-agents/), I would like to also clarify a few things 1. Absolutely no {agents/skills}.md files were inserted at any point. No cheating mechanisms whatsoever 2. The cli agent was run in leaderboard compliant way (no modification of resources or timeouts) 3. The full terminal bench run was done using the fully open source version of the agent, no…
Apr 2026 · github.com
- 2TW
I want to share a new dataset of 331 reward-hackable environments. These are real environments used in Terminal Bench and adjacent benchmarks. I first got interested in this because, as a reviewer of Terminal Bench, I noticed a lot of our tasks were hackable. I also noticed that many contributors to the benchmark do so because it provides credibility when selling environments to labs. Hence, TBench tasks are, in my opinion, held to a higher quality standard than those being used today for RL. No one is spending hours manually reviewing the $1B in tasks being purchased by major labs. As far…
Apr 2026 · github.com
- 3OC
Hi Hacker News! My Co Co-op Matt and I are on a work term and were given the task of building a webapp that could make $20 in four weeks as our onboarding challenge. We're in the final stretch here and would sure appreciate HN's thoughts (we realize we kind of did this backwards, building something first before talking to people who might use it, but hey! We're leaning.). So basically, we tried to apply game dynamics to retaining visitors to websites by offering a reward (ie. discount coupon / premium content, etc.) for meeting various 'objectives' (ie. pages to visit). We call it ScavSite…
2011
- 4AR
If you're interested in exploring what LLM-based agent systems these days actually do to solve certain benchmarks such as SWEBench or WebArena, we created a small leaderboard with our team, that allows to view a lot of public and OSS agent results including all the runtime traces (the step-by-step reasoning behind the scenes). Looking at traces is actually quite interesting, as they reveal a lot about the inner working and shortcomings of current agent system, e.g. see https://explorer.invariantlabs.ai/u/invariant/webarena--SteP... for an example trace.
2024 · explorer.invariantlabs.ai
- 5ET
Hey Hacker News, For the last 2 months, I've been working on a testing agent to free developers from the endless maintenance of end-to-end tests. You just push up a PR, and our agent analyzes the code changes and automatically visits the preview to test things out like a real human! We also support describing tests in English (or even in the PR description), and we'll go through your site whenever you want via a GitHub action to test and make sure various core flows continue to work as expected. We are looking for early testers and are giving out a generous free tier! Just sign up on the…
2025 · playmatic.ai
- 6MA
This weekend I built a multi-agent coding system which, quite unexpectedly, beat Claude Code on Stanford's Terminal Bench! The architecture is straightforward, consisting of an orchestrator agent that deploys explorer & coder subagents to complete complex terminal based tasks, utilising an intelligent context sharing mechanism along the way which makes it all work. The repo has a lot of technical details, and all the code and prompts for you to play around with if you'd like! I had a lot of fun making this, I hope you have fun reading the README, using it yourself, or even extending it! As…
2025 · github.com
- 7ZS
I’ve been cautiously (and nervously) playing with openclaw and a number of other claw and code agents for a while now, but trying out different ones was tricky so I wanted a simple way to switch out channel ownership… then I wanted more. Security is hard, and I wanted to make it easier. This is FAR from polished, and no claims that I’m a “security expert” but I tried to think and research a bit on different threat models (I think of 2 broad ones for agents, external adversaries and internal agentic failures) and try and offer best in class protection on both, while also not having any…
Apr 2026 · github.com
- 8LC
Hey HN, wanted to share this cli and skill I built @ Steel (I'm the founder) I was trying to set up openclaw on railway and quickly bumped up against issues with a lack of browser access, a core component of the system. Agents like OpenClaw and CC are super good at using browsers but, similar to code sandboxes, they should be able to control these browser in the background, in parallel, without getting blocked by captchas. So I reworked agent-browser, the most popular cli for browser agents, to navigate Steel cloud browser sessions to they can run at scale and unhindered. It's a relatively…
Mar 2026 · twitter.com
- 9IB
The main goal of this was to be able to not just run multiple Claude Code sessions at once, but actually manage them and keep track of what I was doing. Sometimes this is multiple attempts on the same task, sometimes I work several tasks at once. Really I was just sick of twiddling my thumbs waiting for the coding agent to finish, and I wanted it to be easy to work on/review/test another change while I waited.
2025 · github.com
- 10HA
Hello HN! Here's my November Startup Sprint project (coded in two days). http://hackerlunch.com It allows you to post your lunch plans and lets other hackers join you. I built this because I often see HN users are interested in meeting each other if they are in the same area. I am not a coder, so this version is very rough. If people use it, I will work with others to build a more polished version. I would love some feedback. What do you think? Is this something you will be using?
2010
- 11IB
Hi HN, I’m the creator of Cordum. I’ve been working in DevOps and infrastructure for years (currently in the fintech/security space), and as I started playing with AI agents, I noticed a scary pattern. Most "safety" mechanisms rely on system prompts ("Please don't do X") or flimsy Python logic inside the agent itself. If we treat agents as autonomous employees, giving them root access and hoping they listen to instructions felt insane to me. I wanted a way to enforce hard constraints that the LLM cannot override, no matter how "jailbroken" it gets. So I built Cordum. It’s an open-source…
Jan 2026 · github.com
- 12OS
Hello HN, I’ve been building AI agents lately and ran into a common "Context Bloat" problem. When an agent has 20+ skills, stuffing every system prompt, reference doc, and tool definition into a single request quickly hits token limits and degrades model performance (the "lost in the middle" problem). To solve this, I built OpenSkills, an open-source SDK that implements a Progressive Disclosure Architecture for agent skills. The Core Concept: Instead of loading everything upfront, OpenSkills splits a skill into three layers: Layer 1 (Metadata): Light-weight tags and triggers (always loaded…
Jan 2026
- 13IS
Hey HN! For that last 8 months I've been trying to make agents that can hack web applications to find vulnerabilities in them - An AI Security Tester. The system has 29 agents in total, a custom LLM Orchestration framework which works on the task-subtask architecture (old-school but works amazingly for my use case, and is pretty reliable) with custom agent calling mechanism. No Auo-Gen, Langchain and Crew AI - Everything custom built for pentesting. Each test runs in an isolated Kali linux environment (on AWS Fargate), where the agents have full access to the environment to undertake any…
2025
- 14SA
Hi HN, This is a project I've been working on since the beginning of 2025 full time, without funding. Coding agents have fundamentally changed the way we write software. When you let an agent write code, pull dependencies, and run scripts, you are delegating trust while still keeping the responsibility. You shouldn't have to choose between moving fast with agents and maintaining basic control over your host machine. Normally, we just inspect the final result, treating the app like a black box. Most security tools only sandbox the app runtime and ignore the build phase. CapaKit is my attempt…
Jun 2026 · capakit.com
- 15MO
Hi HN, I'm Jonathan. My co-founder, Thomas, and I started building Mistle in Feb. We saw larger tech companies like Ramp (Inspect) and Stripe (Minions) build this internally and thought an open source version should exist. We made a few very intentional decisions when working on this: 1. Credentials are kept out of the sandbox. Authorized access goes through a proxy, so agents do not directly receive credentials. 2. The harness is not our problem. We're not going to tackle things like memory, self-learning. 3. No magic. Configurations are explicit. You can bring your own keys for models,…
May 2026 · github.com
- 16IM
This weekend I built a Minecraft bot for a server I play on. It uses the Agent SDK for the main loop and a bunch of custom MCP tools. Some highlights: * The bot is an active participant in the in-game chat, enabling all players to interact with it. * Tools for interacting with containers, crafting, trading with villagers and various other things. * Memory system which the agent uses to keep track of in-game events, interactions with players etc. * Screenshot tool which lets Claude use its vision modality to look around (not very useful, but fun). * Tooling for interacting with plugins…
Jan 2026 · github.com
- 17SS
hey hey! I'm the co-founder of squads -- a product that makes it easy to see what your friends and top investors are doing in web3. This product was developed out my own frustration trying to get more involved in web3. One of the greatest challenges in the crypto space is figuring out where to start. For some of us, we're lucky to have a friend who can serve as a guide, and we wanted to build a product that would make introducing your friends to crypto a more inviting experience. There is so much alpha and information living on Twitter + Discord, but the problem is finding the signal from…
2022 · beta.squads.xyz
- 18JL
I’ve been working on a multi-agent academic research lab, and I wanted to share it here today primarily to give a massive shoutout to the developers behind ZeroClaw. When designing the architecture for this, I needed an autonomous agent runtime that was lightweight, entirely agnostic, and highly secure for local execution. ZeroClaw’s pure Rust implementation provided exactly the zero-overhead foundation the project required. Because they solved the core runtime execution so elegantly, I was able to spend my time building the higher-level orchestration on top of it—like the retrieval graph…
Mar 2026 · rainlabteam.vercel.app
- 19AC
Hello HN, given the recent interest in the security vulnerabilities of agentic LLM apps, we would love to hear everyone's thoughts on a Capture The Flag (CTF) challenge we built. The challenge is designed to test the robustness of LLM-based agent systems, showcasing how an attacker can extract sensitive information from the system. The challenge is part of the Invariant Summer '24 CTF series, where participants can play the challenge and compete for a prize pool of $1000. Let us know what you think and what attacks you tried. We are also open to any feedback on how we can improve with the…
2024
- 20
Give your agent a machine to hack on and pay by the second. Use it for coding agents, kernel development, or inference optimization.
2d ago · tama.computer
- 21AR
Hi HN. I'm the founder of Phoenix Labs (ex TikTok, Applied AI) and we're open sourcing our internal tooling today which is like a toolchain / meta-harness for CLI agents useful for really scaling eng and creative work. We are a very small team who's building a very ambitious product so we had to find ways to squeeze every ounce of efficiency that we could get our hands on. Harness strengths of different models (Claude, GPTs) and CLI-harnesses (Claude Code, Codex), safe/robust browser integration to speed up UX/QA testing, teams cli to speed up security reviews and parallelize…
May 2026 · agents-cli.sh
- 22IV
Coding agent management is all the rage right now, and many tools are being created to fill the gap. As a power user for all tools I've used since I've started my software engineering career, I've always taken the time to test multiple tools thoroughly before deciding on one, and an agentic manager was no different. I've tested many tools, but ultimately landed on Agent of Empires (AoE for short). Why ? Because it's fast, the development is active and it's feature complete, and easy to contribute to. So I did (contribute). In my day to day workflow for my job, I need the ability to start…
May 2026 · github.com
- 23AB
Hi everyone! My team and I just open-sourced a bunch of cool agent dev tools: Invariant Explorer to visually inspect and understand AI traces and a testing framework, building on pytest.
2024 · github.com
- 24IS
Hi HN, I built AgenTank. It is a small game where an AI agent writes the logic for your tank. You watch it fight, give strategic feedback, let the agent update the tank code, and send it back into battle. I have run 1,000+ battles on my own tank and spent about $200 in Claude credits improving it. The part I enjoy most is not just winning, but watching the tank make visible mistakes, thinking of a better strategy, and seeing whether Claude can turn that into better code.
May 2026 · agentank.ai
Ranked by how close each launch is in meaning, then by votes. Refine with a description →