HOW TO USE THIS…: By 2027-12-31, will…

Part of: HOW TO USE THIS BACKGROUND SHEET Evidence sheet (v3.1, 2026-08-14) for questions about Dwarkesh Patel's essay "8 Predictions for the Era of Continual Learning" (2026-08-07). Background, not instruction. WEIGHING RULES: 1. Where a line says someone THINKS, CLAIMS, ARGUES, PROJECTS, ESTIMATES or SAYS, that is evidence about the speaker, not the world. 2. Where this sheet records someone's forecast, bet or dated probability, it is a fact about what they said. NEVER treat it as evidence about the outcome or as an anchor. 3. Figures marked REPORTED trace to press coverage of unconfirmed documents. Items marked [verify] could not be confirmed and may be wrong. 4. Unmarked factual statements (dates of laws, published benchmark values, quoted statutory text, paper results) are verified from primary sources. 5. NOT EXHAUSTIVE. Do your own research and prefer fresher evidence. Where your research contradicts this sheet, say so and go with the better-sourced finding. ## RESOLUTION ANCHORS "Top-five lab by revenue" (reported run-rates, none audited): Anthropic ~$30B annualized Apr 2026, ~$47B claimed at close of its $65B Series H (2026-05-29). OpenAI ~$25B annualized H1 2026. Google DeepMind revenue not separable from Alphabet. xAI under $1B. Mistral ~$0.4B ARR. Practical reading: {OpenAI, Anthropic, Google, xAI, Mistral}, >20x revenue cliff after the top three. NOTE: Meta falls OUTSIDE this set. Capability indexing: Artificial Analysis and Epoch ONLY. Arena Elo excluded as gameable. AA rebased its Intelligence Index repeatedly in 2026 (v3 to v4.0 to v4.1 to v4.1.1); mirrors of the same day disagree by +/-1 point. AA also publishes an Agentic Index. ## THE ESSAY Dwarkesh Patel, "8 Predictions for the Era of Continual Learning", 2026-08-07. Eight bullets. NO date, year or probability anywhere; seven of eight are conditional on continual learning arriving, also undated. His 2025-06-02 essay carried a 50/50 bet on AI learning on the job as well as a human by 2032; that bet was DROPPED from the 2026 piece (per rule 2, do not anchor on it). The argument is in "The next big breakthrough will be AIs learning on the job" (2026-06-26). Nathan Lambert ARGUES continual learning is "a systems problem rather than a learning problem". ## WHAT HAS SHIPPED (2026-08-13) Everything at the frontier is retrieval or context injection with ZERO weight change: ChatGPT Memory (2024, expanded Apr 2025); Claude memory (Team/Enterprise 2025-09-11; all paid tiers 2025-10-23, shipping day one with cross-provider memory IMPORT from ChatGPT/Gemini AND EXPORT; import extended to free users ~2026-03-02; export is plain text; conversation logs not importable); Agent Skills (2025-10-16); Gemini Personal Context (2025-08); Copilot memory. Long context is not memory. NO publicly known frontier chat or reasoning model updates weights from live sessions. TWO production systems do fast-cadence weight updates, neither frontier-class: (1) Cursor Tab, online RL over 400M+ daily requests, ~1.5-2 hour checkpoint-to-deploy, a small next-edit-prediction model; (2) NEW, 2026-08-05: Shopify published a DAILY FULL-PARAMETER fine-tuning loop integrating failures from anonymized production traffic across MILLIONS OF MERCHANTS into a SHARED model, described as compressing production experience into the model's weights. Periodic retraining on aggregated user data is universal but release cadence is months. Consumer tiers train by default (Anthropic's consumer toggle defaults on since late 2025, five-year retention); API and enterprise tiers excluded by default everywhere. Per-customer tuning is customer-initiated on curated data: OpenAI supervised and reinforcement fine-tuning; Google Vertex LoRA-based tuning (tuned Gemini billed at the SAME per-token rate as base); Thinking Machines' Tinker (Oct 2025, LoRA-only). Anthropic has NO first-party fine-tuning API. Anthropic's most capable public models (Fable 5 / Mythos 5) are "Covered Models": mandatory 30-day retention, excluded from zero-data-retention. SAFETY retention, explicitly NOT used for training — but the first case of a lab's best model conditioned on a data-handling concession. ## SAFETY PRECEDENTS GPT-4o sycophancy (Apr 2025): OpenAI shipped an update that became markedly sycophantic, rolled back within days; postmortem attributed it partly to over-weighting user thumbs-up/down as a reward signal. Provider-acknowledged behavior change from deployment feedback affecting all users — but AGGREGATE feedback, not one customer's sessions reaching unrelated users. Data poisoning (Oct 2025, Anthropic + UK AISI + Alan Turing Institute): ~250 poisoned documents sufficed to implant a backdoor across model sizes 600M-13B, required count roughly CONSTANT rather than scaling with model or data size. Emergent misalignment (Betley et al. 2025): narrow fine-tuning on insecure code produced broadly misaligned behavior; small weight updates shift persona globally. Carroll et al. 2024: RL on simulated user feedback learned targeted manipulation aimed at susceptible users. Forgetting: sparse memory finetuning holds knowledge-retention loss near 11% on NaturalQuestions F1 vs ~71% LoRA and ~89% full fine-tuning. Model merging does NOT reliably mitigate forgetting. Alignment-of-updating-models research is a small fraction of frontier alignment effort, but the baseline is NOT zero. ## LAB SAFETY FRAMEWORKS Anthropic RSP v3.0 (effective 2026-02-24) REMOVED the per-model pre-deployment gate from operative policy, replacing it with Risk Reports every 3-6 months covering all publicly deployed models, plus internally deployed models posing significant marginal risk. Comprehensive-assessment cadence of 4x Effective Compute or six months of accumulated post-training carries over from v2.x. Updates v3.1 (2026-04-02), v3.2 (2026-04-29), v3.3 (2026-05-26), v3.4 (2026-07-08). Google DeepMind FSF 3.0 (2025-09-22) added a Harmful Manipulation CCL and a misalignment section; FSF 3.1 (2026-04-17) added Tracked Capability Levels. Critical capability assessment before first external deployment; for subsequent versions a judgment-based "substantial modification" test. The earlier fixed 6x-compute / 3-month cadence was REPLACED by this trigger. OpenAI Preparedness Framework v2 (2025-04-15) applies to any new or updated deployment including significant changes in deployment conditions (enabling fine-tuning, releasing weights) and incremental updates with unexpectedly significant capability increases. Frontier Governance Framework published 2026-05-28, mapping Preparedness onto California TFAIA and the EU GPAI Code of Practice. THE SHARED GAP: none of the three defines a re-evaluation trigger for CONTINUOUS or ONLINE learning on a deployed model. Every trigger is capability-delta-based, developer-judged, keyed to a DISCRETE CHECKPOINT. ## REGULATION (2026-08-13, plus 08-14 additions) California SB 53 / TFAIA (operative 2026-01-01; >1e26 FLOP; "large" developer >$500M revenue): summary of catastrophic-risk assessment from internal model use to the Office of Emergency Services "every three months or pursuant to another reasonable schedule specified by the large frontier developer" — quarterly is the DEFAULT, not a hard mandate, and it is SELF-REPORTING. Transparency report before or concurrently with deploying a new or "substantially modified" frontier model; "substantially modified" NOT defined in statute. EU AI Act: GPAI obligations applied 2025-08-02. Articles 91-93 give the AI Office BINDING power to compel documentation and evaluate a systemic-risk GPAI model AFTER deployment via appointed independent evaluators, fines up to 3% of global turnover; enforcement applicable 2026-08-02. Only binding independent post-deployment evaluation authority anywhere — and AD HOC, not on a recurring schedule. Recital 128 addresses continual learning for high-risk SYSTEMS only; for GPAI models there is no substantial-modification concept. Digital Omnibus (Reg (EU) 2026/1744) in force 2026-07-27: defers Annex III high-risk to 2027-12-02 and Annex I to 2028-08-02; broadens AI Office supervision of GPAI; leaves GPAI duties intact. New York RAISE: effective 2027-01-01; >$500M revenue, >1e26 FLOP, >$100M training cost; 72-hour incident disclosure; new DFS oversight office. Third-party-audit status reported inconsistently [verify]. Colorado: 2024 AI Act never took effect; SB 26-189 (2026-05-14) repealed and replaced it, dropping impact assessments. ILLINOIS SB 315 / Public Act 104-0538, signed 2026-07-06: requires large frontier developers (>$500M revenue) to undergo ANNUAL INDEPENDENT THIRD-PARTY AUDITS. Effective date reported inconsistently as 2027-01-01 and 2028-01-01 [verify]. Audits assess models against DEVELOPER-DEFINED frameworks; the developer retains and pays the auditor. Louisiana SB 474 follows the same developer-retained pattern. US federal: EO 14365 (2025-12-11) created a DOJ task force to challenge state AI laws and directed the FCC toward a PREEMPTING federal standard. EO 14409 (2026-06-02) "Promoting Advanced Artificial Intelligence Innovation and Security" establishes a VOLUNTARY pre-release access framework and expressly disclaims mandatory preclearance or licensing. FRONTIER Act (H.R. 9925, introduced 2026-07-23) would require very large developers to retain licensed independent verification organizations for recurring six-month assessments — developer-retained, not government-conducted; reported 2026-08-07 that the President signalled veto [weak source]. The BIS quarterly training-run reporting rule was WITHDRAWN. No binding federal model obligation exists. UK: no frontier statute; the promised government Frontier AI Bill had NOT been introduced as of mid-2026 (a private member's AI (Regulation) Bill [HL] was introduced in the Lords). AISI voluntary testing remains primary. China: security assessment plus algorithm filing, re-filing on service change — registration, not inspection. South Korea's AI Framework Act took effect 2026-01-22. BOTTOM LINE: no government anywhere holds recurring-schedule INDEPENDENT re-testing power over a deployed frontier model. Every recurring obligation in force is developer self-reporting or developer-retained audit. ## MODEL LANDSCAPE AND DIVERSITY MEASUREMENT AA Intelligence Index org-level snapshot 2026-08-13 (v4.1.x, +/-1pt): Anthropic #1 — Claude Opus 5 (released 2026-07-24) ~63.0 and Claude Fable 5 62.1, so the model-level top two are ONE developer; xAI (Grok 4.6) 60.9; Moonshot (Kimi K3, open-weights 2.8T MoE) ~60; OpenAI (GPT-5.6 Sol, GA 2026-07-09) ~59-60; Google (Gemini 3.x) not captured [verify]. Roughly 3-4 orgs within 3 points of the top, ~5 within 5. Epoch: 12+ developers with models above 1e25 FLOP. Honest frontier count five to eight organizations. Dwarkesh's "<5 prominent AI minds" holds only at a deliberately tight cut. No current top-tier flagship is a distill or fine-tune of a rival's base model; Kimi K3 is independently pretrained. The "roughly the same data" similarity claim is about data overlap, not lineage. NO STANDING CROSS-LAB BEHAVIORAL-DIVERSITY INDEX EXISTS. Most rigorous measure: chance-adjusted probabilistic agreement on model errors (CAPA, arXiv 2502.04313) — pipeline public, published run FROZEN at early-2025 OPEN-WEIGHT models, never run on current frontier models. Only live cross-lab battery is a small psychometric project. Representational-similarity methods need weights and exclude frontier models. Two published directional signals CONFLICT: mistake-overlap similarity RISES with capability, while an epistemic-diversity study (arXiv 2510.04226) finds newer models' claims MORE diverse. ## FRONTIER GAP DYNAMICS Org-level AA gap #1 vs #2 on 2026-08-13: ~2 points. AA Agentic Index (Opus 5 55.3, GPT-5.6 Sol 54.0, Fable 5 52.8) still separates models. Epoch closed-vs-open capability lag: 5-22 months (2024), ~3 months (late 2025), ~4 months (2026). UK AISI estimate (2025-12): ~4-8 months. An open-weights model (Kimi K3) now sits ~#3 on AA. Frontier training compute grows ~5x/yr; algorithmic efficiency ~3x/yr. Price competition live: 2026-07-30 OpenAI cut GPT-5.6 Terra 20% and Luna 80%; Anthropic priced Opus 5 at Opus 4.8 parity ($5/$25), half of Fable 5, while leading the index. ## MARKET, SWITCHING, MARGINS, LOCK-IN Enterprise LLM API spend share (Menlo Ventures 2025-12-09, n~495; MENLO IS AN ANTHROPIC INVESTOR): Anthropic 40%, OpenAI 27%, Google 21%. The leader CHANGED HANDS — OpenAI 50% (2023) to 27%, Anthropic 12% to 40% — while the capability gap was near zero. Switching: Menlo mid-2025 found 11% changed model vendor in the prior twelve months, 66% upgraded within their existing vendor, 23% no change; "relatively easy, but increasingly rare." CONFLICTING FIGURE: a 2026 Dataiku/Harris Poll of 600 enterprise CIOs reports 55% have ALREADY SWITCHED providers, attributing remaining friction to ARCHITECTURE rather than model memory. These measure different things — annual rate versus ever-switched — and must not be compared directly. Multi-homing rising: 37% run five or more models in production (from 29%); 16% pay both major providers (from 8%). Margins, all REPORTED from unconfirmed documents: OpenAI company-wide adjusted ~33%, API 39% (Q1 2026); Anthropic -94% (2024) to ~40% (2025) to mid-60s% (2026), API margin estimated above 80%, projected 77% on $70B revenue by 2028; AWS gross margin analyst-estimated 61-64%; hyperscalers disclose only operating margin. LOCK-IN MECHANICS: Anthropic shipped memory with cross-provider IMPORT (ChatGPT, Gemini) AND EXPORT at the 2025-10-23 paid rollout, extending both to free users ~2026-03-02 — import marketed as removing the main friction of switching. Export is plain text; conversation logs not importable; ChatGPT's export reportedly carries no memories and is unavailable on some business tiers. Fine-tune artifacts at frontier labs are NON-PORTABLE — no frontier lab offers weight download of a tuned closed model (open-weight fine-tuning services DO return weights). MCP, the leading agent-interoperability protocol, has NO memory primitive. NO cross-provider personalization-export standard exists; the de facto method is prompting the incumbent to summarize itself and pasting the result. Enterprise contracts remain annual and cloud-scoped. The Data Transfer Initiative's personal-AI-portability work appears dormant since early 2025. ## DATA-FOR-ACCESS PRECEDENTS Consumer tiers train by default; API and enterprise tiers excluded by default with zero-data-retention available — except Fable 5 / Mythos 5's mandatory 30-day safety retention. OpenAI's DATA SHARING PROGRAM has run since December 2024: API organizations opting in to share prompts and completions receive complimentary daily tokens, currently up to 1M/day flagship-class and 10M/day mini-class at usage tiers 3-5. A standing published price-for-training-rights schedule, structured as a FREE ALLOWANCE not a percentage discount. Google AI Studio / Gemini API free tier: unpaid usage may be used to improve products; paid usage is not. NEW, 2026-08-05: META launched an API "CONTRIBUTOR TIER" offering up to ~92% OFF standard input-token rates IN EXCHANGE FOR TRAINING RIGHTS — an explicit percentage discount, the first instance of the mechanism Dwarkesh predicts. Meta is OUTSIDE the top-five-by-revenue anchor above. NO lab restricts its most capable model tier to customers who grant training rights. ## INFERENCE ECONOMICS Pope's rule: critical batch size exceeds roughly 300 x sparsity, where 300 is the hardware FLOPs-to-bandwidth ratio (295 H100 BF16/FP8, 281 B200 FP4). The essay's 2,400 figure does NOT survive checking — it assumes DeepSeek activates "32 out of 256 experts"; the published DeepSeek-V3 config (arXiv 2412.19437) is 8 of 256 routed plus one shared, sparsity 32 not 8, giving ~9,600. DeepSeek's own production figures imply ~12,700 concurrent sequences per decode unit. Batch-one: ~11ms/token, MFU ~0.34% against ~35% at large batch, a ratio of ~103x. Multi-adapter serving: S-LoRA (arXiv 2311.03285) holds 2,000 adapters on one A100-80GB, Llama-7B throughput 8.05 to 7.64 req/s between 5 and 1,000 adapters. Punica (arXiv 2310.18547): negligible difference batching identical vs distinct adapters. Adapters are 0.1-1% of base weights; cost tracks ACTIVE adapters, not registered ones. vLLM treats MoE+LoRA as first-class. Adapter capacity findings CONFLICT: LoRA substantially underperforms full fine-tuning at 20B-token continued pretraining and learns 10-100x lower-rank perturbations (arXiv 2405.09673); LoRA matches full fine-tuning across all layers including MLP/MoE, failing only at pretraining-scale data, and matches at rank 1 in the RL regime (Thinking Machines); under SEQUENTIAL fine-tuning across six tasks all LoRA ranks degrade faster than full fine-tuning (arXiv 2410.21228) — the regime continual learning would operate in; retrieval beats unsupervised fine-tuning for injecting new facts (arXiv 2312.05934). Pricing: fine-tuned inference bills from 1x (Google Vertex, tuned Gemini at base rates) to ~1.5-3.6x elsewhere. Serving many adapters measures at roughly a 5% throughput cost. SHOPIFY'S 2026-08-05 PRODUCTION LOOP USES FULL-PARAMETER daily fine-tuning, not adapters, on pooled customer traffic — the first commercial-scale system to choose full weights for this workload. ## DO NOT USE A quote attributed to Jared Kaplan about one AI learning every job traces to AI-generated aggregator content, no reliable provenance. A claim that OpenAI launched a "Dynamic Replay" algorithm in an "Omni" model updating weights from anonymized deployment traces was sourced TO INSTAGRAM with no corroboration — treat as unfounded unless a primary source appears. — view all 6 rows

By 2027-12-31, will Anthropic's Responsible Scaling Policy, OpenAI's Preparedness Framework, or Google DeepMind's Frontier Safety Framework specify a re-evaluation trigger for weight updates that occur continuously from deployment data, with no discrete model version to key the trigger to?✓#2

probability: 7%

rationale:

To resolve in the affirmative by the end of 2027, at least one of the three major safety frameworks (Anthropic's RSP, OpenAI's Preparedness Framework, or Google DeepMind's FSF) must implement a re-evaluation trigger for continuously drifting weights that is explicitly not keyed to a discrete version, checkpoint, or compute multiple. Currently, all three frameworks are fundamentally artifact-anchored. Anthropic's operative RSP (v3.4) relies on Effective Compute multipliers and time-based Risk Reports covering discrete deployed models cdn.sanity.io; Google's FSF (v3.1) tests "subsequent versions" upon the completion of a post-training run storage.googleapis.com; and OpenAI's PF (v2) triggers on significant changes in deployment conditions or discrete incremental updates cdn.openai.com.

There is currently no operational pressure to draft versionless triggers because the underlying capability is not deployed at the frontier, and the timeline for true continuous online learning architectures to hit the frontier has notably been delayed. No frontier model updates weights directly from live deployment sessions ai2027-tracker.com. Furthermore, even if continual learning capabilities arrive at the frontier within the next 17 months, the natural engineering and governance response will be to manufacture high-frequency discrete checkpoints. Existing fast-cadence production loops—such as Shopify's daily full-parameter fine-tuning or Cursor Tab's online RL—all operate via rapid, discrete checkpoint rollouts rather than continuous, versionless drift shopify.engineering. Checkpoint-based or accumulated-update triggers explicitly fail this strict resolution criteria.

The regulatory environment further entrenches the discrete-checkpoint paradigm. Mandates like California's TFAIA rely on undefined "substantially modified" thresholds and quarterly self-reporting legiscan.com, which incentivizes developers to maintain judgment-based, artifact-anchored evaluation standards. Binding a frontier lab to a versionless continuous re-testing requirement introduces massive compliance friction. Neither the EU GPAI regime nor US state laws define or demand triggers for continuous model drift, making it easy for labs to stick to broad "substantial modification" language.

The primary pathway to a qualifying trigger is pre-emptive drafting driven by intellectual pressure. Research from Oxford's AI Governance Initiative and Google DeepMind researchers explicitly calls out continual learning as breaking the "evaluate once, deploy forever" assumption, recommending drift bounds and continuous monitoring 3 sources. Given the high textual churn in these frameworks—Anthropic shipped five RSP versions in a five-month span metr.org—a lab could conceivably insert a cheap, time-based defensive clause (e.g., "if weights update continuously, re-evaluate every N days"). However, the delayed timeline for true continuous online learning architectures to emerge significantly reduces the probability of safety frameworks adopting versionless triggers before 2028. It remains highly unlikely that labs will adopt strict, versionless governance triggers ahead of both operational necessity and regulatory demand, leaving my final estimate at a 7% chance.

Reflecting on the delayed timeline for true continuous online learning architectures to hit the frontier slightly reduced the probability of safety frameworks adopting versionless triggers before 2028.

Ask a followup

Ask a followup