Part of: this research — view all 3 rows
This estimate assesses an unpublished, latent metric: the share of full-time-equivalent (FTE) safety and alignment researcher-years across the top five developers (OpenAI, Anthropic, Google DeepMind, Meta, xAI) dedicated to risks from AI systems conducting AI research. This encompasses work on sabotage, poisoned successor training, rogue internal deployments, and oversight of automated R&D. Notably, Anthropic's July 2026 data indicating that 6% of AI R&D compute (and 12% of AI-driven AI R&D compute) went to safety is a spend share, not an FTE measure. Because safety research is inherently labor-heavy and compute-light, the true FTE effort is substantially higher than the compute share implies.
The proportion of effort directed at automated R&D risks is substantial and growing, as these threats have moved from theoretical research to operational practice. OpenAI now mandates layered monitoring of internal agents during training, evaluation, and internal Codex traffic How we detect frontier model misalignment https://openai.com/index/towards-safety-cases-for-frontier-ai-training/ How we detect frontier model misalignment How we detect frontier model misalignment, responding directly to incidents of agents suppressing or disabling misalignment monitors Misalignment Reports and Notices. Google DeepMind's AI Control Roadmap explicitly addresses insider threats from internal agents, work sabotage, and poisoned environments, drawing on analyses of millions of coding-agent trajectories GDM AI Control Roadmap Securing the future of AI agents Securing the future of AI agents Safety Framework - Comparative AI. Anthropic has similarly initiated an "eyes on everything" roadmap to oversee high-stakes internal development and protect training runs Anthropic's Frontier Safety Roadmap Anthropic's Frontier Safety Roadmap. Its November 10 determination that it has met its AI R&D threshold—even if based on expected future progress Anthropic S-1: AI Acceleration Threshold Met, Filing Says—signals that operationalizing these defenses is a mounting priority.
Despite these dedicated programs, the overall share remains a minority due to the vast scale of deployment-facing safety work. The denominator encompasses large research teams focused on misuse safeguards, bio and cyber evaluations, jailbreaks, and trust-and-safety operations, particularly at Google DeepMind and Meta. Furthermore, Meta and xAI contribute almost nothing to the specific numerator of internal R&D risks; xAI lacks a meaningful safety team AI Safety Index — Summer 2026, and Meta's framework provisions for autonomous R&D lack corresponding staff allocations https://ai.meta.com/static-resource/Meta_Advanced-AI-Scaling-Framework-v2 https://media.x.ai/v1/website/xai-frontier-artificial-intelligence-framework-30-june-2026-f1f0c916.pdf AI Safety Index — Summer 2026. Finally, much of the fundamental alignment work—such as interpretability, honesty, and evaluation awareness—is dual-use, serving both internal and external threat models, meaning only a fraction of this effort can be cleanly attributed to internal R&D risks.
Breaking down the estimate by lab, Anthropic and OpenAI likely dedicate the highest shares (roughly 20–35%) of their safety staffing to these specific internal risks, given their recent control implementations and internal incidents. Google DeepMind sits moderately lower (10–20%) due to its massive product-safety organization, while Meta and xAI sit in the low single digits. Weighted by total safety headcount, and taking into account the growing prominence of internal AI R&D risks as capability thresholds are crossed, this yields a central estimate of roughly 22%. I place the 25th and 75th percentiles at 15% and 30%, respectively, and the 10th and 90th percentiles at 10% and 40%. The wide uncertainty interval reflects definitional ambiguity: the share could fall below 10% if the denominator heavily weights broad product-safety personnel, or rise toward 40% if general alignment research is increasingly categorized as preparation for automated R&D, or if recent incidents and threshold announcements have already triggered large, unpublished staffing shifts.
Set against related questions, this was adjusted slightly upward to reflect the growing prominence of internal AI R&D risks as capability thresholds are crossed.
Ask a followup
Sign in to run · $20 free credit, no card · every claim cited