Introducing Decision Forecasts

Short version

Decision makers want to know what their choice will cause.
Conditional forecasting questions assess correlations, not causality.
By narrowing to a use case where four assumptions hold: a sole decider, full context, a decision made soon, and a decision that's hard to unmake, our empirically validated AI forecaster produces causal forecasts like a human. We call them decision forecasts.
They remove non-causal reasoning for applications like grantmaking.

The problem

Imagine you're a senior policymaker deciding whether to back a legislative pause on frontier AI training. You want to know what your decision would do. I put myself in that position and asked Claude Opus 5 for P(B|A) and P(B|not A), where (A) is a US legislative pause on frontier training before the end of 2026 and (B) is more than $1B in damages attributed to an event primarily caused by AI. Its forecasts were:

P(AI disaster | safety legislation passes) ≈ 62%

P(AI disaster | no safety legislation) ≈ 8%

Most people would interpret the forecasts to imply that a legislative pause makes a billion-dollar AI catastrophe roughly eight times more likely.

However, Opus 5's rationale makes clear that it believes nothing of the kind. Instead, it forecasts a 1.5% chance that AI safety legislation is passed and then reasons:

Conditioning on [AI safety legislation] means conditioning on a world where something snapped the political system hard enough to flip an accelerationist executive and a Republican Congress into the most aggressive intervention available, in a matter of weeks. That's not a world you reach through gradual policy drift. It's overwhelmingly a world reached through a shock.

Opus 5's mental model is shock → legislation and shock → damages. Its 62% forecast isn't about what a pause would do; it's about how a world with a pause comes about. Personally, I think it is a well reasoned conditional forecast. The problem is not the forecaster, it is the question.

For a decision maker, Opus 5's forecast is worse than useless because it actively misleads users who may not realize that its key consideration is correlation, not causation.

We set out to fix this.

This is an old problem

Metaculus and Manifold users know that the gap between two conditional forecasts reflects correlation, not necessarily causation. Sometimes the correlation is what you want. And when you want causation, occasionally you can frame a pair of questions carefully enough that the forecasts mostly reflect causation. But that framing work is skilled, manual, easy to get wrong, and often highly specific to a particular situation.

Dynomight's 2022 Prediction market does not imply causation catalogs the fixes required to get a causal answer: getting the causal structure right, randomizing, waiting for a natural experiment, using time, and controlling for confounders. Each of these fixes is partial. The scariest failure mode is the silent one, where a decision maker thinks they are getting a causal forecast but isn't.

We don't have a fully general solution for causal forecasting. We do think we have something that works for use cases like grantmaking. We get there by imposing four assumptions.

Four things that have to be true

Decision forecasts are causal forecasts that require four things to be true: (1) someone specific is deciding, (2) the decider supplies the context, (3) the decision is soon, and (4) the decision is hard to unmake. My observation from looking at many failure modes over the years is that when a conditional forecast fails to reflect causal considerations, the violation of at least one of these is usually the root cause.

Let's briefly walk through the four assumptions and how they lead to failures:

Someone specific is deciding. If a decision belongs to a committee rather than to a single person, then as a forecaster I need to think about the incentives and dynamics of committee members, how that might cause delays, etc. I quickly start reasoning about the state of the world that leads to a decision, rather than the impact of the decision.
The decider supplies the context. For many questions, it is difficult to add value unless the forecaster has context.
The decision is soon. If you aren't deciding soon, then to forecast I need to reason about how the world will evolve until a decision is made.
It's hard to unmake. Decisions that can be reversed tomorrow force me as a forecaster to think through complicated chains, like "If legislation is not passed this month, then it is likely to be passed next month, or the month after…"

Our approach is not grounded in theory but in practice. My inspiration is that somehow human forecasters are able to offer causal forecasts in conversation without making Bayes nets in their head. So we ought to be able to make our AI forecaster do the same.

What a decision forecast does

We start with our validated AI forecaster as the core. We give it a forecasting outcome to forecast and a set of interventions provided by the user, like "I will donate $0, $100, or $1,000 to an organization." Then we add the following additional context:

Imagine you are in the room with the decision maker and will influence the decision. They are deciding soon. Forecast the outcome under each alternative as a separate hypothetical, all things considered. All downstream consequences of the decision count, including other agents' reactions to it (for example, a public funding decision signals conviction and can pull in follow-on funders). The rest of the world responds realistically in every branch. Do not hold everyone else's behavior fixed: others may adapt to the decision, partially offsetting it or amplifying its effects.

That's it. No do-calculus, no graph, no variables to control for. If a user doesn't like our default context, they can add additional context to get a different answer.

For instance, maybe they are considering an anonymous donation. If so, then their reputational benefit will not enter into the downstream impact.

A mistake that I previously made was to think of a causal forecast of a donation as singular and well-defined. It isn't. Are we talking about the direct impact? Are we considering follow-on funding effects? Do I plan to help advise the grantee? All of these are legitimate causal questions.

The additional context above is meant to put the forecaster in the mindset of an adviser and to head off the four failure modes. Based on the reasoning we have observed, it appears to work as well as a great human forecaster providing advice. Unlike a human adviser it is available anytime, costs a lot less, and doesn't get tired.

Case study: grantmaking

My colleague Robert recently published a case study applying decision forecasts to grantmaking. The entire post is worth a read.

A quick summary: A grant maker aiming to maximize the impact per dollar must review potentially dozens or hundreds of diverse applications. First, the grant maker must provide context, often in the form of a grant application, in order for our forecaster to provide useful guidance. If a grant maker is using our app, it will help create a forecasting question around the outcome that the applicant is trying to achieve. The grant maker must also specify what interventions they are considering, for example: no funding, partial funding, full funding. Then our decision forecaster will output how much the grant will causally impact the outcome in expectation.

Our hope is that this allows grant makers to make better, more informed decisions across many possible grantees.

Why not just use an LLM?

Decision forecasts have two key advantages over a raw LLM.

The forecaster underneath is validated. You can see its track record here.
The causal framing is applied by default. The silent failure mode has effectively been removed, so users don't have to worry they're getting a misleading correlation.

We don't have a general causal solution, but we don't think raw LLMs do either.

Summary

Decision forecasts provide the causal impact of decisions similar to a trusted human adviser.

Our AI safety legislation example shows how conditional forecasts can actively mislead about causal relationships, since an accurate conditional forecast might be dominated by correlational considerations rather than causal effects. (Note: We didn't fix this example.)

Decision forecasts require four conditions to hold in order for the forecast to reflect causality. Despite this it works well for real applications like grantmaking that meet all four almost by construction: one decider, deciding this cycle, with the application in front of them, and once the money is wired it isn't coming back.

We have been guided by practical rather than theoretical consideration. We hope you find it useful.