Question
What was the training compute of the largest training run completed by any developer between 2025-08-01 and 2026-11-14?
Anchor and Candidate Runs Epoch's estimate for GPT-6 Astra provides the most solid anchor for this assessment: approximately 9.4e26 FLOP, assuming at least 100,000 GB200s for about 90 days at 25% utilization https://epoch.ai/models/gpt-6-astraGPT-6 Astra. However, this estimate functions more as a floor than a ceiling. We are assessing the true latent maximum across all developers, and most closed frontier models released since mid-2025—including Google's Gemini 4 Argon and Carbon, Anthropic's Claude Fable 5.2, OpenAI's GPT-6.1 series, and Meta's Muse Wonder—lack public compute estimates. Taking the maximum across these highly competitive, concurrent projects pushes the expected largest run comfortably above the 1e27 FLOP mark.
Hardware Realities and Competing Compute Demands While the theoretical capacity to run vastly larger models exists—such as xAI's Colossus 2 reaching roughly 1.1 million H100-equivalents Colossus 2 | AI data centers—it is highly unlikely that any lab has successfully allocated its entire fleet to a single monolithic run. Hardware arithmetic dictates that ~100,000 advanced GPUs running for three months yields roughly 1e27 FLOP GB200 NVL72 | NVIDIA. However, final training runs face intense internal competition for resources. Labs are aggressively splitting their clusters to support inference, reinforcement learning, and parallel experiments, a strategy aligned with the observation that there is an increasing benefit to doing more work at smaller scales. For context, final runs represented only about 9.6% of OpenAI's 2024 R&D compute spending Final training runs account for a minority of R&D compute ....
Pacing of Frontier Scale-Ups A mechanical extrapolation of the historical 5× annual growth trend would imply a maximum well above 3e27 FLOP by late 2026. Yet the actual pacing of single largest runs has been more deliberate, growing only about 2.5× over the 18 months from GPT-4.5 to GPT-6 Astra. Developers are encountering friction and prioritization shifts: Anthropic delayed pre-training its new frontier base model by more than a month for security reasons Anthropic discloses month-long training pause, AI security ..., and xAI's Grok 4.8 was still in reinforcement learning well into the fall Grok 4.8 Release Date: Soft Rollout This Week, Musk SaysGrok 4.8: 2.5T Params, C++ Stack, RL Next (Sept 2026), making it unlikely to count as a completed run by the mid-November cutoff. Nevertheless, consistency with related questions favors steady extrapolation rather than treating the high internal allocation to AI R&D experiments as a halt to growth in final-run compute. Balancing continued scaling against these constraints puts my median at 1.8e27 FLOP, with the central half of the distribution spanning 1.2e27 to 2.9e27 FLOP.
Key Uncertainties The lower bound of the distribution reflects the possibility that Epoch's Astra estimate is overly aggressive—for instance, if the run utilized BF16 precision rather than FP8, the true compute could fall closer to 5e26 to 7.5e26 FLOP. Since the target is the maximum across developers rather than Astra alone, I place the 10th percentile at 8e26 FLOP. Conversely, the wide upper tail extending beyond 4e27 FLOP, with a 90th percentile of 4.5e27 FLOP, accommodates the very real possibility of a quiet, massive run. Google’s Gemini 4 Argon was described as their "most ambitious pretraining run yet" Gemini 4 Brings Google Back into Frontier Competition as ... and utilized Google's immense TPU fleet, while OpenAI has had an unreleased, highly capable model in training since late August. If any lab managed to sustain a 200,000+ accelerator run for three to four months with heavy post-training reinforcement learning, it would readily hit the upper ranges of this estimate.
Set against related questions, the median was raised slightly to reflect steady extrapolation despite the high internal compute allocation to AI R&D experiments.
Ask a followup
Sign in to run · $20 free credit, no card · every claim cited