Part of: this research — view all 6 rows
Resolving this requires computing chance-adjusted probabilistic agreement on model errors (CAPA) across the five highest-scoring organizations' flagship models. However, this measurement has never actually been run on frontier models; the published CAPA pipeline remains frozen at early-2025 open-weight models 2 sources. Since there is no recorded baseline from August 2026, any assessment depends on what a comparable retrospective evaluation would show. Consequently, the probability relies on understanding the structural properties of the CAPA metric and the underlying trends in frontier model behavior.
The strongest evidence suggests that model mistakes are becoming more alike, not less. Research using the CAPA instrument demonstrates that as model capabilities increase, their average error agreement with other developers' models also rises 3 sources. A July 2026 study corroborated this, finding that agreement rises with scale and recency, with cross-lab 2026 flagships exhibiting substantially higher agreement than 2024 models arxiv.org. This convergence is driven by shared training data, distillation practices, and common benchmark optimization. Furthermore, CAPA adjusts for chance agreement associated with accuracy arxiv.org; as model accuracy approaches saturation on standard evaluations, the remaining errors naturally concentrate on the same hard, ambiguous, or mislabeled items, pushing the overlap metric higher.
The primary mechanism theorized to drive divergence—models developing unique characteristics by continually learning from varied deployment experiences—is not operating at the frontier. Evidence indicates that no frontier flagship model updates its weights from live sessions. Recent memory features from major developers rely entirely on context injection or retrieval, not weight updates. While fast-cadence weight updates exist in smaller, specialized production systems, current safety frameworks and API roadmaps indicate no imminent rollout of online learning for deployed frontier reasoning models. Without this mechanism, the structural pressures toward convergence remain dominant.
The most plausible path to a lower CAPA stems from changes in the composition of the top five organizations. In August 2026, the capability spread among the top-five leaders was narrow, spanning roughly 3 points. Under the recent version 5 index update, the gap between the leading model and the fifth organization is about 18 points. Because models in different capability classes tend to have lower error overlap, this widened spread could mathematically pull the mean pairwise agreement down even if the leading models converge. Additionally, an epistemic-diversity study indicates newer models generate more diverse factual claims arxiv.org, though this measures open-ended generation rather than CAPA's specific construct of benchmark error overlap.
Ultimately, the instrument-specific evidence strongly points toward continued convergence as frontier models scale and optimize against shared targets. While the shifting composition and widening capability spread of the top-five cohort present a credible path to a lower mean CAPA, the structural drivers of error agreement keep the probability of divergence relatively low.
Ask a followup
Sign in to run · $20 free credit, no card · every claim cited