rationale:

Forecast Summary & Core Dynamics I estimate the token count required to reach a fixed capability level will fall by a median factor of 4.6x between 2026 and 2031, with an interquartile range of 2.4x to 9.5x and a wide 80% confidence interval ranging from 1.3x to 22.0x. Evaluating this question alongside related estimates of compute efficiency doublings and structural data constraints reinforces the median while reflecting, particularly in the interquartile range, a fundamental tension between theoretical algorithmic limits and practical developer behavior. Parallel expectations of a 2.8x increase in human data spending suggest labs will rely somewhat more on brute-force data than pure algorithmic efficiency, modestly compressing the distribution. No published series currently isolates frontier sample efficiency from overall compute efficiency Will we run out of data to train large language models?Algorithmic progress in language models. If compute efficiency improves by 1.6x to 3x per year and savings are split evenly between parameters and tokens in a Chinchilla-style regime, the required tokens would fall by roughly 1.25x to 1.7x annually, yielding a 3x to 15x reduction over five years.

Downside Risk: Inference Optimization and Rising Empirical Token Counts The lower tail of this distribution accounts for significant measurement and resolution risk. In practice, actual token counts at the frontier are increasing, not decreasing. Open-model tokens per active parameter have grown approximately 3.1x a year, from ~10 in 2022 to ~300 in 2025 Training open-weight models is becoming more data ..., and recent frontier models like DeepSeek V4 use up to 32-33T tokens DeepSeek V4 Guide: Engram Memory, Training Data .... Labs deliberately overtrain smaller models on massive datasets to minimize downstream inference costs. If the resolving credible estimate relies heavily on empirical training runs rather than compute-unconstrained benchmarks, the observed sample efficiency could appear relatively flat, keeping the ratio near 1.0. Furthermore, increased reliance on synthetic data and reinforcement learning rollouts could inflate the counted tokens needed to reach a given capability.

Upside Potential: The Data Wall and Small-Scale Breakthroughs Conversely, the upper tail is driven by the impending depletion of high-quality public human text, expected between 2026 and 2032 Will we run out of data to train large language models?. This data wall creates immense pressure to invest heavily in data efficiency Will we run out of data to train large language models?. Small-scale experiments indicate that large gains are theoretically possible, though they are often "paid for" with extra compute Dissolving the Deep Learning Sample Efficiency Gap. For example, Kim et al. achieved a 5.17x data efficiency improvement via ensembling at 200M tokens Pre-training under infinite computePre-training under infinite compute, the NanoGPT Slowrun achieved a 10x improvement NanoGPT Slowrun: 10x Data Efficiency with Infinite Compute, and q0 demonstrated a ~13x improvement on validation loss q0: Primitives for Hyper-Epoch Pretraining. Additionally, Patel and Han's experiments suggest data improvements have historically contributed more to compute efficiency (12.0x) than model improvements (3.7x) over a six-year period Pretraining progress is mostly coming from dataPretraining progress is mostly coming from data.

Synthesis Ultimately, the 4.6x median anticipates that targeted research into data curation, synthetic data filtering, and distillation will yield meaningful structural reductions in the unique tokens strictly required for a given capability level. However, realizing the massive theoretical gains (10x to 100x) seen in small, compute-heavy experiments at the frontier scale remains highly uncertain. This assessment carefully balances the structural incentives for better data efficiency against the observational reality that labs are currently willing to spend vastly more pre-training tokens to produce cheaper, smaller inference models.