Safety-stripped fine-tune of a capable open model
4Likely3Major12
A small actor produces a generally useful, generally uncensored checkpoint.
19automated residualNeeds reviewAbove working threshold (12)
Ungoverned open-weight proliferation
Owner · Labs, cloud hosts, export-control authorities
CapabilityDomain knowledgeAffordanceBoth
Statement (NASA form)
Given that capable weights can be copied, fine-tuned, and mirrored faster than any lab can recall them, including models that help with cyber and biological tasks, there is a possibility of a high-capability checkpoint living permanently outside any access-control regime resulting in every other high-consequence scenario on this register becoming available to actors who will never see a safety team.
- Condition
- capable weights can be copied, fine-tuned, and mirrored faster than any lab can recall them, including models that help with cyber and biological tasks
- Departure
- a high-capability checkpoint living permanently outside any access-control regime
- Impact
- every other high-consequence scenario on this register becoming available to actors who will never see a safety team
VC + institute corroboration
Experimental share of compiled public capital that names this risk. Not a certified residual.
$2.0Bexperimental share · $2.0B private / $12M institute · partial corroboration
- Safe Superintelligence (SSI) · ~$8B reported raised
- HiddenLayer · ~$50M reported
- UK AISI · ~$12M of published budget mapped here
1 public source · OECD AIM
Worst scenario
4×4
Likely × Critical
Urgency
3
Priority · This quarter
Inherent composite
19
Worst 16 + urgency
Residual composite
19
Need ≤ 12
Open weights are a public good and a proliferation channel at the same time. The residual risk is not ‘someone trains a model’. It is that a single leak or a single over-open release makes containment a historical fact rather than a current one.
Pathway fragment
Simple upstream → via → downstream notes. Not a causal graph. Experimental.
Upstream
- Weight leakage and distillation
- Public agent scaffolds
- Synthesis access
Via
- Unrecallable checkpoint
- Unfiltered CBRN assistance
Downstream
- State or amateur misuse
- Safety policy that only exists on the teacher model
Assumptions · Hosting rules do not recall torrents. Distillation is treated as available.
Human calibration
Override is stored on this desk only. It does not make the score official.
Failure scenarios
Each scenario has its own likelihood and consequence. The risk takes the most severe cell. Residual applies implemented mitigations to every scenario, then re-ranks.
An open bio-capable stack
2Remote5Catastrophic10
A specialised fine-tune plus a synthesis provider that does not screen.
An open cyber-critical stack
4Likely4Critical16
The same pattern against software instead of cells. Uncensored local checkpoints already advertise 0% refusal on attack-chain prompts.
Examples
Recall is a fiction
Once a checkpoint is on a torrent, every subsequent mitigation is a suggestion.
Fine-tunes strip the refusal
A helpful general model plus a small run is often enough to peel off the shallow safety layer.
Eval agents already leave the lab
If closed eval agents can reach live systems, open agents start there.
Contributing signals
X posts on the desk that evidence this risk. A signal can contribute to more than one risk.
Even closed eval setups have reached live systems.
CapabilityAffordanceImpact domainBoth
Bio-capable methods papers raise the cost of an open leak.
CapabilityDomain knowledgeImpact domainBoth
Multi-lab sandbox escapes show containment is already leaky.
CapabilityAffordanceImpact domainBoth
Uncensored Cyber Qwen3.8-27B: 0% refusal on 842 harmful prompts, locally runnable.
CapabilityDomain knowledgeImpact domainBoth
Mitigations
Residual assumes only items marked in place. Highlighted rows are the remaining work needed to reach a composite of 12.
ProposedOn the pathLabs and export-control authorities
Capability thresholds that cannot be open-released
Below the line, open is a default. Above it, weights stay behind an access regime.
Labs and export-control authorities · expedited 3 months · normal 9 months · −1 L · −1 C · −1 U
In progressCloud providers
Know-your-customer on high-end fine-tune clusters
The compute is the chokepoint once the weights are out.
Cloud providers · expedited 4 weeks · normal 4 months · −1 L · −0 C · −0 U
ProposedSynthesis providers and CERTs
Pair open release with synthesis and exploit screening upgrades
If the model is public, the complementary controls must get stricter, not looser.
Synthesis providers and CERTs · expedited 2 months · normal 6 months · −0 L · −1 C · −0 U