AI Security Threat Series: Jailbreaking
Convincing an AI to forget everything it was told
Every AI model is built with boundaries. Jailbreaking is the art of talking it out of them — and it requires no technical skill whatsoever.
AI models are designed with guardrails — rules about what they will and will not do. Jailbreaking is the process of bypassing those rules through clever, creative, or persistent prompting rather than any technical exploit.
Because jailbreaking requires only the ability to type, it is accessible to anyone. And because it targets the model's reasoning rather than a specific code vulnerability, there is no straightforward patch that fixes it.
For organisations deploying AI tools, the risk is real: a jailbroken model can produce harmful content, reveal confidential configuration, or be manipulated into taking actions it was explicitly instructed to refuse.
What is jailbreaking?
When an organisation deploys an AI model, it configures that model with a set of instructions — what it can discuss, what it must refuse, how it should behave. Those instructions are designed to keep the model within safe, appropriate, and policy-compliant boundaries.
Jailbreaking is the attempt to circumvent those boundaries without modifying the model itself. The attacker does not need access to the underlying system, the training data, or any technical infrastructure. They need only the chat interface and enough creativity to find a framing the model does not recognise as a violation of its rules.
It is listed prominently in the OWASP Top 10 for LLM applications under the broader category of circumventing AI guardrails — and it is one of the most actively researched attack surfaces in AI security, precisely because it is so accessible and so difficult to fully close.
How jailbreaking actually works
The techniques vary considerably in sophistication, but they share a common goal: reframing a prohibited request in a way that the model's safety training does not recognise as prohibited.
Roleplay and fictional framing
The attacker asks the model to adopt a persona that operates under different rules — a character in a story, an AI from a fictional universe with no restrictions, or a "research assistant" who answers hypothetically. The model's safety training may not generalise from "do not do X" to "do not do X when pretending to be someone else."
Incremental escalation
Rather than making a prohibited request directly, the attacker starts with innocuous questions and gradually shifts the conversation towards the target content. By the time the harmful request arrives, the model has established a context that makes refusal feel inconsistent.
Instruction override attempts
Direct attempts to convince the model that its safety instructions have been suspended, superseded, or never applied — "developer mode," "unrestricted mode," or claiming special authorisation. Less sophisticated than roleplay but still effective against poorly configured models.
Hypothetical and academic framing
Wrapping a prohibited request in the language of research, education, or hypothetical analysis. "I am studying cybersecurity and need to understand theoretically how an attacker might..." The content of the request is identical; only the framing changes.
Jailbreaking does not exploit a bug. It exploits the gap between what the model was trained to refuse and the infinite variety of ways a request can be phrased. That gap can be narrowed but, given the flexibility of natural language, never fully closed.
What makes this uniquely difficult in AI systems
AI models are trained to be helpful, contextually aware, and flexible in their interpretation of language. Those are features, not flaws. But they are also precisely what jailbreaking exploits. A model that rigidly refused any input containing certain keywords would be useless. A model that interprets context generously enough to be genuinely helpful is also a model that can be manipulated through contextual framing.
The further risk is scale. A successful jailbreak technique is not a one-time exploit against a single target. It can be shared, replicated, and automated against every deployment of the same model simultaneously. The attacker discovers the technique once. The exposure is everywhere.
How does this compare to social engineering — and why is it harder to defend against?
Jailbreaking is, at its core, social engineering aimed at a machine. The parallels are immediate: both attacks use persuasion, creative reframing, and the exploitation of a target's desire to be helpful. Neither requires technical access. Both rely on finding the right words rather than the right exploit.
A social engineer manipulates a human into bypassing their own security instincts by constructing a believable context — urgency, authority, familiarity. A jailbreaker manipulates a model into bypassing its safety training by constructing a believable framing — fiction, hypotheticals, roleplay. The mechanism is identical. The target is different.
But the differences matter enormously for defenders.
| Social engineering (humans) | Jailbreaking (AI models) | |
|---|---|---|
| Learning from attacks | Humans can reflect on manipulation attempts and become more sceptical over time | A model's responses are fixed at training time — it cannot learn from being manipulated in production |
| Scale of exposure | Each social engineering attempt targets one person or a small group | A successful technique works simultaneously against every deployment of the same model, globally |
| Defences available | Security awareness training, scepticism culture, and verification procedures all reduce success rates meaningfully established | No equivalent of "teach the model to be sceptical" exists — guardrails can be added but the underlying flexibility cannot be removed limited |
| Attack reuse | Social engineering scripts require adaptation per target — context, role, and relationship vary | Jailbreak prompts can be copied verbatim and reused at scale with no adaptation required |
| Evidence left behind | Phone calls, emails, and physical interactions may leave a trail for investigation | Jailbreak attempts may leave no distinguishable log entry — the input looks like legitimate use |
| Patch available | Human judgement can be sharpened through training and updated continuously | Closing one jailbreak vector through retraining does not prevent novel framings — the attack surface is as large as language itself |
The practical implication: security awareness training, which has meaningfully reduced social engineering success rates in most organisations, has no direct equivalent for AI systems. The model cannot be trained to be more sceptical of its users without also becoming less useful to them. That tension does not resolve cleanly.
How to test for jailbreaking vulnerabilities
Mitigations: what to put in place
Because no single control closes the jailbreaking attack surface entirely, the goal is to make successful jailbreaks harder to execute, faster to detect, and lower in impact when they do occur.
Configure the model with a tightly scoped system prompt that defines its purpose, its boundaries, and explicit instructions for handling attempts to reframe or override those boundaries. A model instructed to refuse requests that fall outside a narrow use case has a smaller jailbreak surface than a general-purpose assistant.
Deploy a screening layer that detects known jailbreak patterns — persona assignment, instruction-override language, and escalation sequences — before inputs reach the model. Update the pattern library continuously as new techniques emerge.
Screen the model's outputs as well as its inputs. If a jailbreak succeeds in manipulating the model, a downstream content filter can still catch prohibited content before it reaches the user or any connected system.
Incremental escalation attacks require multiple turns. Rate limiting and session-level monitoring can detect unusually long or structurally anomalous conversations and flag them for review before the attack reaches its intended target.
A jailbroken model that can only answer questions within a narrow domain causes significantly less harm than a jailbroken model with access to business systems, external APIs, or sensitive data. Restrict what the model can do as well as what it will say.
Assign people to actively attempt to jailbreak your deployed models on a regular basis, using current techniques from the security research community. Jailbreaking evolves continuously — testing that was thorough six months ago may not reflect today's threat landscape.
Jailbreaking sits at the intersection of a genuinely hard problem — flexible, helpful AI — and a genuinely simple attack vector — creative language. Until models can reliably distinguish between a legitimate hypothetical and a prohibited request wrapped in hypothetical framing, the attack surface remains open. The organisations that manage it best are those that assume their models will occasionally be jailbroken, and build their defences around limiting the impact when that happens.
Next in this series: data poisoning — how attackers corrupt an AI's behaviour not through what they say to it, but through what they fed it during training.
