Technical posts from the FutureSearch team.
Tom LiptayThe question groups and weights we will use to score ForecastBench's October 25 round, published before any question resolves. Correlated questions share a group and get lower weights under the Metaculus AI Benchmarking formula.
Tom LiptayWe're competing in ForecastBench's October 25 round and will publish our own analysis and leaderboard. Our methods, set out in advance: Brier scores, lower weights for correlated questions, scoring only questions that have reached their scheduled resolution date, and bootstrapped confidence intervals.
Tom LiptayRobin Hanson proposed markets on the share price if the board fires the CEO versus keeps them. We asked our AI forecaster the causal version for eight large companies, on three-year earnings and on total-return share price, with a repeat run and a control. Firing costs earnings everywhere by a similar small amount, the price effect is concentrated in a few companies, and the question formulation matters as much as the forecaster.
Tom LiptayConditional forecasts assess correlations, not causality, and can badly mislead decision makers. We introduce decision forecasts: causal forecasts from our empirically validated AI forecaster, built for use cases like grantmaking where one person is deciding soon, with full context, and the decision is hard to unmake.
Dan SchwarzFutureSearch forecasts of Google DeepMind after the August 5, 2026 reorganization: Hassabis to chairman, Jeff Dean's Discovery Loop, and a delayed Gemini 4. The forecast put general availability at a June 2027 median; Gemini 4 Argon shipped on October 13, 2026. One further senior departure by February 2027, a 59% chance Anthropic's Google Cloud spend passes Gemini revenue before 2028, and a 24% chance DeepMind's old military redline is crossed by end of 2027.
Jack WildmanIn LiteLLM v1.83.10, the allow_client_tags security feature started silently stripping caller-supplied request_tags unless an admin opts the key or team in. The request still returns 200, with no error and no caller-side signal, so the tags our billing pipeline depends on vanished from LiteLLM_SpendLogs. For six and a half days every paid task and conversation deducted $0. Here is how the tag stripping happened, why our spend alerts missed it, and what we changed.
Dan SchwarzDeepSeek V4 Pro beats GPT-5.5 Pro on benchmarks at a fifth of the cost, but three forecasts put its enterprise API share near 4% by end of 2026, with OpenAI holding $180 pricing and the coding lead.
Dan SchwarzGitHub Copilot's move to per-token billing sparked 'Tokenpocalypse' warnings of a 14x cost spike. A forecast puts the effective cost per seat at ~$19/month, because typical usage fits inside the included credit allowance.
Dan SchwarzA forecast of whether Intel's Crescent Island AI GPU, pitched as an HBM-free Nvidia and AMD rival, lands a major cloud. The answer: a 13% chance any top-four hyperscaler deploys it by end of 2027.
Dan SchwarzMicrosoft declared independence from OpenAI at Build 2026 and is chasing top-four-lab status. A forecast puts its best in-house GPQA Diamond score at a median 86.5% by year-end, just below the ~88-92% frontier band where DeepMind, OpenAI, and Anthropic sit.
Dan SchwarzAI overconfidence gets most of the attention, but expert auditors found cases where an Opus 4.6 forecasting agent derived the right answer, wrote out the math, cited the correct precedent, and then assigned a probability inconsistent with its own analysis. Four case studies from BTF-2.
Dan SchwarzFrontier AI forecasting agents are good at finding historical patterns. They are noticeably worse at noticing when the conditions that produced those patterns have changed. Three case studies from the BTF-2 benchmark show the same failure: agents extrapolate base rates without checking whether the generating mechanism is still active.
Dan SchwarzAI forecasting agents tend to model the most extreme version of an outcome, correctly explain why it's unlikely, and then assign that low probability to the entire question. We identify this catastrophizing pattern across three geopolitical case studies from the BTF-2 benchmark.
Dan SchwarzRunning the same forecasting agent more than once and averaging beats any single run. Ensembling across two Opus 4.6 runs and other frontier models cuts Brier score on 1,367 BTF-2 benchmark questions, and a worked example shows how a second run finds context the first missed.
Dan SchwarzWhen will AGI arrive? See how the AGI timeline forecasts of Daniel Kokotajlo, Eli Lifland, Dario Amodei, Peter Wildeford, and the Metaculus community shifted from 2023 to 2026, and why everyone who updated in early 2026 moved their date sooner.
Dan SchwarzOne year after AI 2027 launched, we revisit the forecasts and compare the scenario's predictions to what actually happened with Anthropic, the Pentagon, and Claude Mythos.
Dan SchwarzA hands on workflow for researching and forecasting Polymarket questions with AI. Start from 100 prediction market questions, forecast them with research agents, pull live order book data from the Polymarket API, and rank markets by how far each AI forecast diverges from the current market price.
The full Claude Code transcript from discovering and responding to the litellm 1.82.8 PyPI supply chain attack on March 24, 2026 — from mysterious process explosions to malware identification to public disclosure.
The litellm 1.82.7 and 1.82.8 supply chain attack on PyPI hit 47,000 downloads in 46 minutes. We analyzed all 2,337 dependent packages - 88% had version specs that allowed the compromised versions.
litellm 1.82.7 and 1.82.8 on PyPI were compromised in March 2026 with a malicious .pth file that steals SSH keys, cloud credentials, and secrets, then spreads across Kubernetes clusters. FutureSearch first reported it to PyPI. Learn which versions to avoid, how to check if you are affected, and what to do next.
A malicious litellm release on PyPI compromised our machine through an MCP server's unpinned dependency. No prompt injection, no LLM trickery, just a poisoned package auto-downloaded by uvx.
Robert GambeeJavaScript's Date.parse() will turn almost any string into a date. We tested V8, SpiderMonkey, and JavaScriptCore with surprising inputs and documented every quirk, from ISO 8601 UTC gotchas to why date-fns parseISO rejects what new Date() accepts.
Daniel HnykWe run an SEO pipeline that reads Google Search Console data, spawns an Opus agent per page, and proposes title and description changes. Each agent reads the history of every experiment we've run on that page. Over time, the suggestions get better.
Robert GambeeDocker-based MCP servers leave behind zombie containers because the Docker daemon keeps them alive after Claude Code exits. Switching from docker run to uvx eliminates the problem entirely.
Peter MühlbacherA practical guide to LLM trace analysis: how we use a Claude Code skill to debug AI agents by reviewing their own traces, catching scaffolding bugs, tool failures, and reasoning errors that manual skimming misses.
Peter MühlbacherWhat Claude 4.6's effort parameter actually controls. For Opus and Sonnet 4.6, effort sets reasoning depth and also how thoroughly the model acts: how many tool calls it makes, how much it cross references, and whether it follows your system prompt. Why effort=low can quietly ignore instructions, and when to bump it to medium.
Daniel HnykA practical guide to running Claude Code as a Kubernetes CronJob. The Dockerfile (Python plus Node), the entrypoint and claude command flags, jq log filtering, timeout safety nets, and the CronJob manifest, with the gotchas we hit running it in production for months.
Daniel HnykHow we replaced Airflow and CI pipelines with Claude Code skills and subagents. Markdown files define multi-step workflows, agents execute each phase, and outputs land as plain files in GitHub.
Tom LiptayWe take our AI forecaster's probability estimates, compare them to live Kalshi order books, and build a simulated portfolio to benchmark whether the forecasts are actually accurate.
Rafael PoyiadziInstead of dumping thousands of rows into the MCP tool response, split the audience: content for the model (text summary), structuredContent for the user (interactive widget at zero token cost), and a download URL for the sandbox.
Robert GambeeOpenAI's Responses and Chat Completions APIs have inexplicable inconsistencies. A real-world example of Conway's Law, where org structure dictates software design.
Rafael PoyiadziHow to let Claude upload large CSVs and files to your MCP server without eating the context window. A step by step guide to presigned URLs, the code execution sandbox, and the artifact ID pattern, covering Claude Code, Claude.ai, and Claude Desktop.
Robert GambeeAnthropic, OpenAI, and Google LLM APIs look interchangeable on paper, but the differences break your code: JSON Schema rejections, structured output mismatches, forced tool calls, and prompt caching gotchas. The concrete quirks we hit running tens of thousands of LLM calls per day across all three providers.
Peter MühlbacherWe benchmarked the cost and speed of deep research agents on Deep Research Bench. Claude 4.6 Opus and Gemini 3 Flash lead, with frontier models from $0.05 to $0.55 per task. See the best accuracy per dollar.
Nikos BosseSemantic deduplication uses AI to catch duplicates that exact matching misses. Learn how fuzzy matching detects entries like "IBM" and "International Business Machines" as the same entity across thousands of rows.
Tom LiptayLearn how to evaluate companies by criteria like founder alignment, moat strength, and capital allocation. Score and compare stocks by any criteria to evaluate and test a custom investment thesis
Robert GambeeWhich companies have had the most C-suite churn over the last 10 years? I researched all S&P 500 companies to find out.
Dan SchwarzWho leads the AI race in 2026? We rank OpenAI, Anthropic, Google DeepMind, Meta AI, and xAI across model quality, data, compute, talent, and R&D. Predictions of Anthropic's rise ahead of their March 2026 surge in revenue
Dan SchwarzSix months after the AI 2027 report predicted a fast AGI timeline, we revisit the forecasts alongside Karpathy's critiques. How have the original predictions held up, and why are timelines shifting?
Dan SchwarzA practical guide to building LLM-powered web research agents, covering search strategies, source evaluation, and synthesis of findings.
Dan SchwarzTom LiptayA critical look at the AI 2027 report's claims about superhuman coding — why the timeline is likely too aggressive.
Tom LiptayDan SchwarzEven American-made products rely on imported components, steel, and aluminum. Our analysis breaks down how tariffs increase the real cost of a Ford F-150 by $2,600-$3,800 and a Tesla Model 3 by $1,900-$2,400, with data on every major input.
Dan Schwarz
Lawrence PhillipsThe Apple and OpenAI Siri partnership has frayed, with OpenAI weighing legal action and Apple now eyeing Google Gemini. See how FutureSearch predicted this breakdown back in June 2024, before the WWDC keynote.
Dan SchwarzWorked examples of how OpenAI, Gemini, and Perplexity Deep Research handle hard web research tasks, drawn from FutureSearch's Deep Research Bench evals. See where agents give up too early (lack of persistence) and where they repeat the same failed search instead of adapting, with concrete CyberSeek and UK excess deaths cases.
Dan SchwarzOpenAI launched Deep Research on February 3, 2025. FutureSearch tested the new tool on six research tasks with known answers and found a jagged frontier of performance: confident wrong answers, odd source choices, and incomplete research that looks comprehensive. Here is what it gets right and where it fails.
Dan SchwarzThe inside story of how prediction markets were built, killed, and revived at Google — published in Asterisk Magazine.
Dan SchwarzLawrence PhillipsA video presentation from Dan Schwarz and Lawrence Phillips on integrating AI into forecasting workflows, with practical approaches and lessons learned.
Dan SchwarzA recap of the forecasting tournament pitting human forecasters against AI bots on Manifold Markets.