Jailbreaking
Definition
Jailbreaking attempts to bypass the security boundaries of a language model through inputs.
Jailbreaking in the language model context considered here refers to attacks intended to cause a security-aligned model to produce undesired outputs. Research investigates, among other things, conflicts between learned objectives and limits of their generalization.
Simply explained
Like trying to fool an entrance control with contradictory instructions. However, a model has no human understanding of rules; its behavior arises from learned patterns.
Why it matters in practice
When introducing, allowed tasks, undesired outputs, and independent evaluation of security tests should be documented separately.
Example from business
Possible example: An authorized test team checks in a shielded environment whether an assistant complies with the established security boundaries.
Limits and pitfalls
- Passing a set of known tests does not prove general resilience. The cited work explains why earlier attack tests do not exclude further errors.
Origin & history
Wei and co-authors analyzed fundamental error patterns of security alignment in language models in 2023.
Use in mid-sized companies
- Possible fields of application are testing internal assistants and preparing their introduction, each with authorized test scope and finding protocol.
Frequently asked questions
Is this referring to hardware modification?
This exclusively concerns bypassing language model security boundaries.
Are known attack tests sufficient as complete proof?
No, the work describes limits of the transferability of such tests.
Sources
- Jailbroken: How Does LLM Safety Training Fail? · Wei et al. (2023): Jailbroken: How Does LLM Safety Training Fail?
Editorially reviewed on: