What was the training compute of the largest training run completed by any developer between 2025-08-01 and 2026-10-16?

The GPT-6 Astra Benchmark The best-documented candidate for the largest completed training run is OpenAI's GPT-6 Astra (released September 2026). Epoch estimates its training compute at 9.4e26 FLOP GPT-6 Astra GPT-6 Astra, which the NYT described as "nearly an octillion" calculations What Are Data Centers Doing? A Mind-Boggling Amount of .... OpenAI reportedly stated that Astra was pretrained on more than 100,000 GB200 GPUs at the Stargate Abilene cluster the first 100000-GPU training run at Stargate Abilene the first 100000-GPU training run at Stargate Abilene. Independent hardware math largely supports a figure in this vicinity: assuming 100,000 GPUs, 2.5 PFLOP/s dense BF16 throughput, 30–40% cluster utilization, and a 100-day run yields roughly 6.5e26 to 8.6e26 FLOP for pre-training A back-of-the-envelope estimate puts GPT-6 Astra at ... A back-of-the-envelope estimate puts GPT-6 Astra at .... Adding post-training reinforcement learning, or assuming FP8 throughput, could comfortably push the total above 1e27 FLOP. However, analyst @scaling01 notes that Astra likely did not monopolize the entire cluster, as other models were trained simultaneously EpochAI also released some estimates for GPT-6 compute ... EpochAI also released some estimates for GPT-6 compute ....

Undisclosed Runs Across Frontier Labs Because the true maximum is the highest value across several frontier labs, the median sits well above Astra's 9.4e26 benchmark. Google, Meta, and xAI possess immense compute pools and have completed runs without public estimates. Google's Gemini 4 Argon, a sparse mixture-of-experts model trained on its TPU fleet Gemini 4 Argon - Model Card Gemini 4 Argon - Model Card, and Meta's Muse Wonder, trained on the gigawatt-scale Prometheus cluster Muse Wonder Returns Meta to the Frontier as Compute ... Muse Wonder Returns Meta to the Frontier as Compute ..., are strong contenders to exceed 1e27 FLOP. Anthropic's Claude Mythos has similarly drawn independent estimates spanning 3.4e26 to 1.5e27 FLOP The 6.1e27 FLOP figure for Claude Mythos is not realistic. ... Analyzing the Mythos Preview Capability Jump China will likely have its own Mythos-like model around .... Meanwhile, xAI’s Colossus 2 cluster boasts ~1.1 million H100-equivalents Colossus (data center), though parameter-heavy models like Grok 4.8 and Grok 5 were likely still undergoing RL through September and may not count as completed runs within the time window Elon Maps the Grok Roadmap Through Grok 5 After 4.8 ... Grok 5: Release Date, Specs & What's Confirmed (September ... Grok 5: Release Date, Specs & What's Confirmed (September ....

Scaling Headwinds vs. Physical Limits While Epoch's historical ~5x/year scaling trend implies a frontier between 5e27 and 1e28 FLOP by late 2026, several structural headwinds pull the likely maximum down. Commercial inference demands now absorb a rapidly growing share of total compute, and labs increasingly partition their massive clusters among concurrent models rather than dedicating an entire site to a single monolithic run. Furthermore, as Greenblatt notes, labs are finding strategic benefits in running more experiments at slightly smaller scales.

Distribution and Key Uncertainties

• Lower Tail (p10: 7.5e26): Acts as a soft floor aligned more closely with documented benchmark runs. This assumes Epoch's 9.4e26 estimate for Astra is somewhat inflated (e.g., due to shorter run durations or lower utilization) and that no other competitor meaningfully exceeded it.
• Lower-Middle (p25: 9.5e26): Sits near the documented Astra benchmark, reflecting substantial probability that the largest completed run was only modestly above—or effectively in line with—that estimate once uncertainty in run duration and utilization is considered.
• Median (p50: 1.35e27): Reflects the high probability that at least one of the major competitors (Google, Meta, xAI) surpassed Astra by leveraging their immense clusters for a dedicated run, while commercial inference demands constrained the scale of monolithic training.
• Upper-Middle (p75: 2.3e27): Allows for a major lab to have sustained a substantially larger dedicated run on part of a gigawatt-scale cluster, without assuming that the cluster was monopolized or that historical scaling trends translated directly into a single completed run.
• Upper Tail (p90: 3.8e27): Assumes a highly utilized, sustained run—potentially using FP8 precision—on a significant fraction of a gigawatt-scale cluster. Unsubstantiated rumors, such as a leaked 6.1e27 FLOP slide for Mythos The 6.1e27 FLOP figure for Claude Mythos is not realistic. ..., are dismissed as falling well outside a realistic 90th percentile bound.