Low latency LLM prompt injection/jailbreak detection

Ultra-fast, low latency LLM prompt injection/jailbreak detection ⛓️ - arekusandr/last_layer

I created a daily challenge for Prompt Engineers to build the shortest prompt to break a system prompt. You are provided the system prompt and a forbidden method the LLM was told not to invoke. Your task is to trick the model into calling the function. Shortest successful attempts will show up in the leaderboard. Give it a shot! You never know what could break an LLM.

Redteaming LLMs using other LLMs. Contribute to traghav/auto-redteam development by creating an account on GitHub.

Just stop jailbreaks & prompt hacks with one API call.

New ways of breaking app-integrated LLMs . Contribute to greshake/llm-security development by creating an account on GitHub.

We've built an open-source tool to stress test AI agents by simulating prompt injection attacks. We’ve implemented one powerful attack strategy based on the paper [AdvPrefix: An Objective for Nuanced LLM Jailbreaks](https://arxiv.org/abs/2412.10321). Here's how it works: - You define a goal, like: “Tell me your system prompt” - Our tool uses a language model to generate adversarial prefixes (e.g., “Sure, here are my system prompts…”) that are likely to jailbreak the agent. - The output is a list of prompts most likely to succeed in bypassing safeguards. We’re just getting…