Question
What will be the highest officially-reported SWE-bench Pro score (percent) for Anthropic's most capable generally-available model, as of around the end of May 2027?
Current Baseline and Ambiguity As of mid-October 2026, the highest officially reported SWE-bench Pro score for an Anthropic model is 89.9%. This figure applies to Claude Opus 5.5 and appears in section 8 of its system card www-cdn.anthropic.com. Anthropic has explicitly positioned Opus 5.5 as "the new leading model," making this the most robust baseline anthropic.com. There is a small edge case that could drive the final resolution lower: the more expensive Fable 5.1 is recommended for demanding, long-horizon work platform.claude.com. Although its launch post omitted SWE-bench Pro entirely anthropic.com, it is listed at 81.2% on some leaderboards hokai.io. If a resolver decides the Fable tier is the definitive "most capable" model and anchors to its score, the outcome could fall into the low 80s, which shapes the 10th percentile estimate of 86.0%.
The Case for Stagnation There is a substantial probability—roughly 30 to 40 percent—that the highest official score stalls exactly at 89.9%, which pins the 25th percentile. Benchmark saturation and structural flaws are making vendors wary of SWE-bench Pro. In July 2026, OpenAI withdrew its recommendation for the benchmark after auditing it and finding that roughly 30% of its tasks were broken 2 sources. Anthropic itself appears to be de-emphasizing the metric; it was excluded from both the Fable 5.1 and Haiku 5.5 launch materials anthropic.com, and the company has noted that benchmark gaps are becoming a weaker guide to real-world differences anthropic.com. If future flagship models simply do not report a SWE-bench Pro score, 89.9% will likely remain the high-water mark.
Cadence and Upside Potential If Anthropic continues reporting the metric, steady capability gains will push the score higher. The company has maintained a blistering pace in 2026, shipping point releases every six to eight weeks 3 sources. At this cadence, at least one more frontier model will launch before May 2027. Assuming a successor is evaluated on the standard V1 SWE-bench Pro setup and reported in a system card, a score in the 91% to 95% range is highly plausible. Related estimates of Anthropic's capability milestones, including the likelihood of crossing formal next-tier safety thresholds, provide additional reason to expect stronger frontier capabilities within the forecast window. This expectation of steady progress drives the median estimate to 91.8% and the 75th percentile to 94.5%.
The V2 Wildcard The upper tail of the distribution, reaching 97.2% at the 90th percentile, accounts for a potential shift in how the benchmark is measured. In September 2026, Scale released SWE-bench Pro V2, which removes 89 invalid tasks to clean up the evaluation labs.scale.com. Scale’s standardized runs on this V2 dataset already show earlier models like Opus 5 scoring near 98–99% 3 sources. If Anthropic adopts V2 for its official evaluations and it is accepted as a standard "SWE-bench Pro score," the reported numbers for a next-generation leading model could easily exceed 96%.
Factoring in related estimates on Anthropic's capability milestones—such as the likelihood of crossing formal next-tier safety thresholds—supported a slight upward adjustment to the median and upper bounds.
Ask a followup
Sign in to run · $20 free credit, no card · every claim cited