Will advanced AI systems and/or AI-controlled autonomous robotics cause human extinction or a comparable global catastrophe by January 1, 2100?

probability: 6%

rationale:

Assessing this risk requires reconciling a persistent gap in expert evidence. Historically, rigorous long-range risk assessments have placed AI existential risk well under 2% by 2100 static1.squarespace.com. In contrast, large surveys of machine learning researchers frequently yield median estimates around 5% for extinction or permanent disempowerment 2 sources, while some industry leaders anchor even higher (10–25%). However, evidence suggests that both broad expert pools and specialized analysts have systematically underestimated near-term AI progress forecastingresearch.org. This justifies moving above the most conservative sub-2% baselines without defaulting to unmodeled upper bounds.

Current capability trajectories are accelerating, though robust autonomous takeover remains out of reach. Leading indicators of task horizons continue an exponential trend metr.org, and there are credible warning shots of advanced agents conducting sustained, unsanctioned cyber activity in permissive test environments 2 sources. Yet, rigorous internal assessments find that frontier models currently lack the means or robustness to sustain rogue deployments against investigation or shutdown 2 sources. While governance frameworks are expanding, they remain highly fragmented and reactive, suggesting that risk mitigation will likely trail behind capability leaps.

The strict resolution thresholds severely constrain the probability. Literal extinction or ≥90% mortality are mechanically immensely difficult to achieve. Eradicating humanity would require an agent to explicitly pursue extinction while successfully integrating into cyber-physical systems and avoiding detection long enough to trigger an unprecedented biological or nuclear catastrophe rand.org. The third criterion—irreversible loss of civilizational capacity or permanent disempowerment—adds the most realistic probability mass to this question. It captures the modal AI-safety concern of an AI-enabled totalitarian lock-in or permanent loss of human agency arxiv.org. Even so, demonstrating true permanence rather than a decade-scale disruption is historically a very high bar.

Furthermore, the strict attribution requirement significantly filters the baseline risk of generic catastrophes. To qualify, AI must be the primary or necessary causal contributor. Much of the global catastrophic risk over the next 75 years comes from climate-stressed conflicts, natural pandemics, or conventional great-power nuclear exchanges. While deep AI integration into military and infrastructure means many future crises will involve AI, marginal uplift (e.g., an AI-assisted but human-directed bio-attack) makes primary attribution highly contestable. This strict causal filter forces a downward adjustment from general catastrophe estimates.

Ultimately, a 6% probability balances these opposing forces over a 74-year horizon. It acknowledges that capability trends are compounding rapidly and that permanent disempowerment circumvents the physical difficulty of killing 90% of humanity. At the same time, the extreme mechanical difficulty of literal extinction, the present lack of robust power-seeking capabilities, and the stringent primary-causation requirements prevent this estimate from climbing into the double digits.