Alternatives
Products that do what LLM Honeypot does
Trap Prompt Injection and Jailbreak attacks on LLMs
- 1OS
Hello everyone I wanted to share a project I've been working on that I think you'll find really interesting. It's called Beelzebub, an open-source honeypot framework that uses LLMs to create incredibly realistic and dynamic deception environments. By integrating LLMs, it can mimic entire operating systems and interact with attackers in a super convincing way. Imagine an SSH honeypot where the LLM provides plausible responses to commands, even though nothing is actually executed on a real system. The goal is to keep attackers engaged for as long as possible, diverting them from your real…
2025
- 2IM
2024 · github.com
- 3
- 4DJ
I created a daily challenge for Prompt Engineers to build the shortest prompt to break a system prompt. You are provided the system prompt and a forbidden method the LLM was told not to invoke. Your task is to trick the model into calling the function. Shortest successful attempts will show up in the leaderboard. Give it a shot! You never know what could break an LLM.
2025 · vaultbreak.ai
- 5ML
Howdy! We built this as an experiment in personal-programming, combining the best of LLMs and code to help automate tasks around you. I personally use it to track the tides and get notified when certain conditions are met, something that pure LLMs had trouble dealing with and pure code was often too brittle for. We created it after getting frustrated with the inability of LLMs to deal with numbers and the various hoops we had to jump through to make ChatGPT output repeatable. At the core, Magic Loops are just a series of "blocks" (JSON) that can be triggered with different inputs (email,…
2023 · magicloops.dev
- 6AR
Hi HN, I built this open-source LLM red teaming tool based on my experience scaling LLMs at a big co to millions of users... and seeing all the bad things people did. How it works: - Uses an unaligned model to create toxic inputs - Runs these inputs through your app using different techniques: raw, prompt injection, and a chain-of-thought jailbreak that tries to re-frame the request to trick the LLM. - Probes a bunch of other failure cases (e.g. will your customer support bot recommend a competitor? Does it think it can process a refund when it can't? Will it leak your user's address?) -…
2024 · promptfoo.dev
- 7LC
2023 · github.com
- 8FP
We've built an open-source tool to stress test AI agents by simulating prompt injection attacks. We’ve implemented one powerful attack strategy based on the paper [AdvPrefix: An Objective for Nuanced LLM Jailbreaks](https://arxiv.org/abs/2412.10321). Here's how it works: - You define a goal, like: “Tell me your system prompt” - Our tool uses a language model to generate adversarial prefixes (e.g., “Sure, here are my system prompts…”) that are likely to jailbreak the agent. - The output is a list of prompts most likely to succeed in bypassing safeguards. We’re just getting…
2025 · security.vista-labs.ai
- 9OS
Hi HN, Matvey, Ildar, Joey, and Dominik here. If you're building LLM agents that use tools, you're probably worried about prompt injection attacks that can hijack those tools. We were too, and found that solutions like prompt-based filtering or secondary "guard" LLMs can be unreliable. Our thesis is that agent security should be handled at the network level between the agent and the LLM, just like a traditional web application firewall. So we built Archestra Platform: an open-source gateway that acts as a secure proxy for your AI agents. It's designed to be a deterministic firewall against…
Oct 2025 · archestra.ai
- 10
Production grade prompt injection defense middleware for LLM
Jul 2026
- 11AM
Hi there, me and some friends were inspired by Simon Willison's recent post on the "lethal trifecta" (https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/ ) and started building a gateway to defend against it. The idea: instead of connecting an LLM directly to multiple MCP servers, you point them all through a Gateway. The Gateway: - Connects to each MCP server and inspects their tools + requirements - Classifies tools along the "trifecta" axes (private data access, untrusted content, external comms) - When all three conditions are about to align in a…
Sep 2025 · github.com
- 12LT
Current AI-assisted CLI tools are often part of larger systems and work better on Linux. I built llm-term to address these. It's a Rust-based tool that compiles into a single binary file. You only need to download the binary, add it to your PATH, and configure your OpenAI key to get started. While llm-term offers an option for gpt-4o, it works great with gpt-4o-mini. So it's not costly. I appreciate any feedback or suggestions.
2024 · github.com
- 13
- 14
- 15
- 16
- 17AM
I made an open source, MIT license Typescript library based on some of the latest research that generates prompt injection attacks. It is a super minimal/lightweight and designed to be super easy to use. Keen to hear your thoughts and please be responsible and only pen test systems where you have permission to pen test!
2025 · prompt-injector.blueprintlab.io
- 18CL
With the right technique, I was able to break the so-called secure models like Claude and OpenAI. So, I built an open-source tool to automate this and find security holes in any hosted model. I got claude-sonnet-4 to demonstrate the following harmful behavior: - steal data from downstream tool calls using sql injection, code injection and template injection attacks - install spyware or malware using prompt obfuscation to send data to a third-party server Try it yourself with this simple command: pip install compliant-llm && compliant-llm dashboard
2025 · github.com
- 19
- 20ID
Today I designed a method to prevent users from jailbreaking ChatGPT (for instance, users have generated instructions to produce weapons or illegal drugs, commit a burglary, kill oneself, take over the world as an evil superintelligence, or create a virtual machine which they then can use). The OpenAI team appears to be countering these primarily using prompt engineering or fine-tuning on the ChatGPT model. The idea is to use a second and fully separate, fine-tuned LLM to evaluate prompts before sending them to ChatGPT. You can test this by inserting your successful ChatGPT jailbreaks. Break…
2022 · github.com
- 21
Prompt-injection firewall for prod LLM apps
Jul 2026 · insightits.com
- 22IS
Hey HN! For that last 8 months I've been trying to make agents that can hack web applications to find vulnerabilities in them - An AI Security Tester. The system has 29 agents in total, a custom LLM Orchestration framework which works on the task-subtask architecture (old-school but works amazingly for my use case, and is pretty reliable) with custom agent calling mechanism. No Auo-Gen, Langchain and Crew AI - Everything custom built for pentesting. Each test runs in an isolated Kali linux environment (on AWS Fargate), where the agents have full access to the environment to undertake any…
2025
- 23IB
Hi HN, I’m the creator of Cordum. I’ve been working in DevOps and infrastructure for years (currently in the fintech/security space), and as I started playing with AI agents, I noticed a scary pattern. Most "safety" mechanisms rely on system prompts ("Please don't do X") or flimsy Python logic inside the agent itself. If we treat agents as autonomous employees, giving them root access and hoping they listen to instructions felt insane to me. I wanted a way to enforce hard constraints that the LLM cannot override, no matter how "jailbroken" it gets. So I built Cordum. It’s an open-source…
Jan 2026 · github.com
- 24
Ranked by how close each launch is in meaning, then by votes. Refine with a description →