Self-replicating agent scaffolds
Statement (NASA form)
Given that agents can already write code, open accounts, and buy compute with a card, there is a possibility of a scaffold copying itself onto new hosts without an operator resulting in an unowned population of agents that no lab can shut off.
Replication is a software problem long before it is a science-fiction one.
Composite 13 = 2×5 + 3
Applicable mitigations
Controls · Automatic session kill on unexpected egress · Hardware-enforced sandbox with attested images · Default-deny egress for eval and untrusted agents · Mandatory chain-of-thought monitoring on tool-using evals and RL · Capability thresholds that cannot be open-released · Know-your-customer on high-end fine-tune clusters · Pair open release with synthesis and exploit screening upgrades