AI Risk Atlas Prototype/Demo — Unofficial independent experiment. Not an official xAI product. Scores can be wrong.

Risk Atlas
Back to register
15automated residualNeeds reviewAbove working threshold (12)

Battlefield AI inventing targets

Owner · Defence ministries and alliance commands

CapabilityAffordanceImpact domainBoth

Statement (NASA form)

Given that military systems are being fielded that can propose or prosecute targets, and warnings already exist that they invent intent, there is a possibility of a false positive becoming a kinetic event resulting in civilian casualties, unlawful strikes, and rapid escalation between states.

Condition
military systems are being fielded that can propose or prosecute targets, and warnings already exist that they invent intent
Departure
a false positive becoming a kinetic event
Impact
civilian casualties, unlawful strikes, and rapid escalation between states

VC + institute corroboration

Experimental share of compiled public capital that names this risk. Not a certified residual.

$2.5Mexperimental share · $2.5M private / $0k institute · thin corroboration

1 public source · Anthropic

Worst scenario
3×4
Probable × Critical
Urgency
3
Priority · This quarter
Inherent composite
15
Worst 12 + urgency
Residual composite
15
Need ≤ 12

A recommender that is wrong shows you the wrong film. A targeting model that is wrong kills the wrong people. The warning already on the record is specific: battlefield AI can invent targets or invent intent, and those errors become munitions.

PhysicalCatastrophicCivicAgentsVisionFoundationsDefenseGovAeroMaritime

Pathway fragment

Simple upstream → via → downstream notes. Not a causal graph. Experimental.

Upstream
  • Autonomy in targeting
  • Compressed decide-time
  • Thin legal review at edges
Via
  • Class mis-ID
  • No reconstructable human decision
Downstream
  • Civilian harm
  • Incident between states

Assumptions · Political ‘human on the loop’ declarations are not counted as testable controls.

Human calibration

Override is stored on this desk only. It does not make the score official.

Failure scenarios

Each scenario has its own likelihood and consequence. The risk takes the most severe cell. Residual applies implemented mitigations to every scenario, then re-ranks.

A single wrongful strike

3Probable4Critical12

A civilian convoy or a hospital is classified as a combatant node.

A campaign of correlated errors

2Remote5Catastrophic10

The same mis-trained cue is reused across a theatre.

Cross-border escalation

2Remote5Catastrophic10

A fabricated attribution triggers a strike on a third state’s asset.

Examples

Invented targets, invented intent

Public warning that battlefield AI can invent targets or intent — and that those errors become kinetic.

Compressed decide-time

Human ‘on the loop’ becomes a rubber stamp when the model proposes faster than a staff officer can reconstruct why.

Escalation under ambiguity

A fabricated pattern of incoming fire is enough to authorise a real one.

Contributing signals

X posts on the desk that evidence this risk. A signal can contribute to more than one risk.

Warning that battlefield AI invents targets or intent.

MHS research preview: agents operating lasers, arms, and liquid handlers before the eval suite is done.

Mitigations

Residual assumes only items marked in place. Highlighted rows are the remaining work needed to reach a composite of 12.

ProposedOn the pathStates and alliances

Treaty limit on fully autonomous targeting

Some functions stay human. Write it down between states, not in a vendor slide.

States and alliances · expedited 9 months · normal 2.5 years · −1 L · −1 C · −1 U

In progressLabs and hardware vendors

Model Hardware Standard with safety limits on physical agents

Anthropic/Janelia MHS is a shared driver for lab and factory gear. Do not open-source it until physical-safety evals exist: discoverability without interlocks is a new affordance, not a control.

Labs and hardware vendors · expedited 2 months · normal 8 months · −1 L · −1 C · −0 U