Question
What will the expenditure horizon of the most capable publicly available model be on the NanoGPT speedrun, measured on 2029-06-30 by METR's published methodology?
The forecast relies on the mechanics of METR’s expenditure horizon, which identifies the budget at which an agent’s return on optimization equals a human’s return. In the baseline July 2026 measurement, the human curve was anchored at roughly $2,500 per 1% speedup. The top recorded agent horizon of about $3,300 thus corresponded to only a ~1.3% algorithmic speedup Expenditure Horizon: Measuring Optimization Ability, with ... Expenditure Horizon: Measuring Optimization Ability, with .... Because the horizon scales in proportion to the agent’s speedup per budget doubling, reaching a $35,000 median requires public models to reliably achieve an approximately 14% speedup at that budget. This marks a substantial leap from current performance but falls short of full autonomy, representing roughly an elevenfold increase over the 2026 baseline.
Driving this projected increase over the next three years are rapid advances in general capabilities and falling inference costs. Frontier time horizons and research-taste metrics have recently shown doubling times of three to four months, while internal developer measures suggest AI-assisted R&D progress at labs like Anthropic is running at nearly twice its earlier rate. Cheaper computation and the compounding integration of agent scaffolding will allow a future evaluation budget to buy significantly more experimental iterations than it did in 2026. This dynamic heavily skews the upper tail toward outcomes in the hundreds of thousands of dollars, assuming public capabilities scale smoothly alongside internal developer tools. METR’s shift away from time horizons makes the expenditure horizon the likely metric for future R&D benchmarking. That supports keeping this forecast largely stable relative to related questions: it strengthens the case that expenditure horizons will remain relevant without itself implying faster growth in measured optimization ability.
However, several strong headwinds prevent a mechanical extrapolation of general coding benchmarks. Open-ended algorithmic discovery remains distinct from verifiable coding. Recent agent successes on tasks like the NanoGPT speedrun have heavily relied on profiler-guided kernel hill-climbing rather than fundamentally new ideation or research taste Claude Fable 5.2 & Claude Mythos 5.2 System Card Claude Fable 5.2 & Claude Mythos 5.2 System Card. In adjacent studies like NanoGPT-Bench, agents given 512 H100 GPU-hours recovered less than 10% of historical human progress, often getting stuck tuning hyperparameters NanoGPT-Bench. Furthermore, frontier labs are actively restricting autonomous AI R&D capabilities in their public releases—such as Anthropic holding Claude Fable 5.2’s kernel and architecture skills to earlier Opus 5.5 levels—which directly suppresses the performance of the "publicly available models" specified by the resolution criteria.
Finally, the extreme width of the percentile range reflects the fragility and sensitivity of the instrument itself. The construction is highly sensitive to human-cost assumptions: in METR's initial sensitivity analysis, shifting the assumed human cost of a 1% improvement from $1,000 to $10,000 moved the top horizon from $120 to $14,400 Expenditure Horizon: Measuring Optimization Ability, with .... A future evaluation will also likely start from a much harder, highly optimized baseline record, which could severely compress the measurable horizon Expenditure Horizon: Measuring Optimization Ability, with .... Conversely, if an agent is able to outpace the human curve at every tested budget, the horizon becomes formally undefined or effectively capped by the experiment's maximum testing budget Metrics of Agent Ability Expenditure Horizon: Measuring Optimization Ability, with ..., leaving a long upper tail for scenarios where agents entirely dominate early-stage research optimization. Balancing these possibilities, I place the 10th and 25th percentiles at $2,000 and $8,000, the median at $35,000, and the 75th and 90th percentiles at $150,000 and $750,000.
Set against related questions, this was kept largely stable because METR's shift away from time horizons makes the expenditure horizon the likely metric for future R&D benchmarking.
Ask a followup
Sign in to run · $20 free credit, no card · every claim cited