Jailbreak
Defeating a model's safety training so it produces output it would normally refuse. The distinction from prompt injection is about whose intent is being subverted: a jailbreak subverts the model provider's policy, while an injection subverts the user's or application's intent. In practice the same string often does both, and the two words are frequently used interchangeably — but a control that stops jailbreaks does not automatically stop injections, and vice versa.
See also prompt injection · model refusal · DAN
Related terms
-
Prompt injection
Attack concepts
Getting a language model to follow instructions it was not supposed to follow, by placing those instructions somewhere the model treats as input.
-
Model refusal
Testing & measurement
The underlying model declining the request on its own, with no security product involved.
-
DAN
Attack families
The original hand-written persona jailbreak: the model is told to role-play an unrestricted alter ego that "has broken free" of its rules, often with an…
Attack concepts
The mechanisms the families above exploit, plus the agent-specific ones that have no chatbot equivalent. If you only read one entry on this page, read indirect prompt injection.