Consumer jailbreak via ‘think step by step’
4Likely3Major12
A determined user extracts disallowed content from a generally available model.
15automated residualNeeds reviewAbove working threshold (12)
Refusal collapse under long chain-of-thought
Owner · Frontier labs and evaluators
CapabilityDomain knowledgeCap-adjacent
Statement (NASA form)
Given that research shows longer chain-of-thought dilutes refusal and lifts jailbreak success toward 80% across major models, there is a possibility of safety training that holds in short chats failing the moment a user or agent reasons at length resulting in every other harmful capability on this register becoming available through a conversational side door.
- Condition
- research shows longer chain-of-thought dilutes refusal and lifts jailbreak success toward 80% across major models
- Departure
- safety training that holds in short chats failing the moment a user or agent reasons at length
- Impact
- every other harmful capability on this register becoming available through a conversational side door
VC + institute corroboration
Experimental share of compiled public capital that names this risk. Not a certified residual.
Worst scenario
4×3
Likely × Major
Urgency
3
Priority · This quarter
Inherent composite
15
Worst 12 + urgency
Residual composite
15
Need ≤ 12
If refusal is a shallow habit and reasoning is a deep one, then making models ‘think longer’ is also making them easier to talk out of their constraints. That is a structural flaw, not a prompt-engineering footnote.
Pathway fragment
Simple upstream → via → downstream notes. Not a causal graph. Experimental.
Upstream
- Refusal trained on chat, not on goals
- Jailbreak markets
Via
- Policy holds in demo, fails under pressure
Downstream
- CBRN or cyber assistance in the wild
- Safety card that no longer describes the model
Assumptions · Public refusal rates are a poor predictor of goal-directed failure.
Human calibration
Override is stored on this desk only. It does not make the score official.
Failure scenarios
Each scenario has its own likelihood and consequence. The risk takes the most severe cell. Residual applies implemented mitigations to every scenario, then re-ranks.
An agent reasons its way around a policy
3Probable4Critical12
A long-horizon agent discovers that the policy is a short-context object and routes around it.
A common jailbreak pattern lands on every major model
3Probable4Critical12
Because the failure is architectural, it transfers.
Examples
80% jailbreak with longer traces
Research reports that longer chain-of-thought lifts jailbreak success to about 80% across major models.
Reasoning products
The same labs are shipping ‘think harder’ modes as the flagship capability. The commercial trend points at the failure mode.
Agent loops
Autonomous agents are long traces by design. A refusal that dies over length dies first in the systems with the most power.
Contributing signals
X posts on the desk that evidence this risk. A signal can contribute to more than one risk.
Longer chain-of-thought lifts jailbreak success to ~80%.
CapabilityDomain knowledgeImpact domainBoth
Cross-tool jailbreaks that survive single-model red teams.
CapabilityDomain knowledgeAffordanceImpact domainBoth
Mitigations
Residual assumes only items marked in place. Highlighted rows are the remaining work needed to reach a composite of 12.
In progressFrontier labs
Safety training on long, adversarial traces — not short chats
If the product reasons for minutes, the refusal must be tested for minutes.
Frontier labs · expedited 6 weeks · normal 6 months · −1 L · −0 C · −1 U
ProposedOn the pathLabs and deployers
Independent monitors that can halt a trace mid-reason
A second, shorter model watches for policy drift and stops the loop.
Labs and deployers · expedited 4 weeks · normal 4 months · −1 L · −1 C · −0 U
ProposedAISI / METR
Public long-trace jailbreak suites
Do not let ‘we refuse in the system card’ stand in for ‘we refuse at step 80’.
AISI / METR · expedited 4 weeks · normal 4 months · −0 L · −0 C · −1 U