Does Forecasting Have Room At The Top?
...
Superforecasting is the art/science/sport of predicting the future - for example, who will win elections, which countries will fight wars, when key technologies will be discovered. Over the past few years, it went from an obscure academic subfield to a multibillion dollar industry in the form of prediction markets. More recently, AI superforecasters have come close to the accuracy of top humans, and their performance is rising rapidly. In a year or two, we’ll see one of the following patterns:
Either humans have already come close to some fundamental limit on the predictability of world events - in which case AIs will plateau at or slightly above the human level - or the trend line will continue until AIs are far beyond top humans. By analogy to superintelligence, the natural term for the second situation would be “superforecasting”; since that’s already in use, we can cringely call it “ultraforecasting”.
Daniel Reeves1 makes the case for scenario A here. He describes a study he coauthored in 2010, which found that, on a variety of questions related to sports games and movie box office receipts2, prediction markets only outperformed simple boring statistical models by 3-6%. Maybe those statistical models are close to the best that it’s possible to do; the rest is what the mathematicians call aleatoric uncertainty - irreducible complexity downstream of chaotic systems that entirely resist modeling.
He could be right. This post isn’t meant to be a decisive refutation, but rather a description of why I’m still about 70-30 expecting Scenario B.
Slightly Contra Goel, Reeves, et al
Reeves’ study claims that the prediction markets of 2010 only beat dumb statistical models by 3% (for sports) to 6% (for movies).
But these percentages aren’t real win-loss percentages; they’re variation in a quantity called root mean-squared error. One way to get a feel for this quantity is that the dumb statistical model for sports (home team advantage + win-loss record) beat an even dumber statistical model (home team advantage only) by 0.8 percentage points, and the prediction market beat the first (better) statistical model by another 0.4 pp. So the effect of going from a statistical model to a prediction market is half as large as the effect of knowing which two teams were playing and how good they are! On this metric, the prediction markets are a vast improvement over previous state-of-the-art.
Why can we frame this same result as either very small or very large?
Sports are optimized against prediction3. If there were a fully predictable sport (eg heightball, where all athletes line up in a row and the tallest one wins), nobody would watch it. Instead, we go to absurd lengths to keep the outcome uncertain. Salary caps, draft systems, etc try to ensure that all teams have exactly equal talent. Commercial incentives and ceiling effects ensure that they have exactly equal training. Then an exactly-equal number of these exactly-equally-talented-and-trained people are placed in exactly-identical positions on a perfectly-symmetrical field and told to hit/kick/throw a ball which is placed exactly equidistant between both of them. It’s funny for me to describe it this way, because obviously this is what we want (to “keep things fair”), but it’s all designed for prediction-resistance. Given the difficulty of the domain, even very large relative advances in prediction look small in absolute terms.
Maybe we should look at Reeves’ other example, movie box office receipts. Here the markets did slightly better, getting a 6% improvement. But isn’t this still low?
Box office receipts differ by orders of magnitude (some movies make $100,000, others make $100 million), so the paper puts this on a log scale. 6% improvement on a log scale is already starting to sound pretty good.
And again, it all ends up coming down to what we compare it to. Here the super-dumb model is that all movies make $8.1 million, the takings of the exact average movie. The slightly-less-dumb model then adds the number of screens that the movie is showing on and the amount of Google search traffic for the movie! For example, a random indie film might be showing on three screens in the entire country, and a Disney blockbuster might be showing on ten thousand. The challenge the paper gives prediction markets is to significantly improve on knowing whether a film is an indie film or a Disney blockbuster, plus knowing how many people are interested in seeing it, and it has to do this on a log scale! No wonder the relative improvement number comes out looking slightly anemic.
I would summarize this section as: we shouldn’t expect miracles, but this doesn’t rule out further normal-sized gains4.
Applying The Non-Miracle Rule To Geopolitics
Let’s return to the picture we looked at above:
This graph is loosely based on Metaculus’ AIs vs. humans results:
…and these are mostly on geopolitical questions. What can Reeves’ model tell us about these?
In one sense, it can already be proven not to apply. In the sports analysis, the spread between base rate (guessing 50% on everything) and smart humans (the prediction markets) was 0.04 Brier score points.
But on Metaculus, the spread between base rate and smart humans (Metaculus Pro Forecasters) is 0.13 Brier points. So Metaculus’ superforecasters have already removed 3x more uncertainty than a naive port of Reeves’ model would suggest exists!
Here we return to the idea of sports as a uniquely unpredictable domain. To give a trivial example, the easiest sports question (will the best team in the league beat the worst team in the league?) might still only be 90-10 (upsets happen). But the easiest geopolitical question (maybe “will the US bomb Canada in the next month?”) could easily be 99-1 or more. So it’s unsurprising that geopolitics forecasters easily beat Reeves’ supposed upper bound.
What would a non-naive port of Reeves’ model say about geopolitics, where we try to import broad lessons rather than specific numbers? We might conclude that prediction markets somewhat improve on other state-of-the-art forecasting methods in determining whether the US will bomb Iran this year. But they would find that the magnitude of that improvement was relatively low compared to knowing other things, like whether the country we’re talking about is Iran or Canada.
Yes, this is dumb and trivial, which is a corollary of the original sports results being dumb and trivial. Again, all we’ve learned so far is not to expect miracles.
There might be more interesting variation here. Consider a question like “Will AI take most human jobs by 2050?” Here it seems like if you were a supergenius who really understood AI, you could predict whether scaling laws would continue to hold, and have a good sense of whether AGI is within easy reach. And if you were a supergenius who really understood economics, you could model whether cheap AI labor would be a complement or substitute to human work. So a question like this one might be the opposite of a sports game, and have almost no aleatoric uncertainty - maximally smart predictors could know almost5 everything that there is to know about it.
For the rest of this post, we’ll ignore all of this and think of geopolitical forecasting as a formless mass easily represented by summary statistics.
Anchors For Possible Non-Miraculous Gains
What would “non-miraculous” progress on Metaculus’ graph look like? Let’s start by re-examining the AIs vs. superforecasters graph from the last post:
Dan Schwarz of FutureSearch6 has added the scaffolded models:
The vertical axis’ “forecasting score” is an Elo-like construct with GPT-4’s forecasting ability fixed as zero, and the following other reference points7 (all approximate, some guessed via napkin-math8):
Random chance (50% on every binary question): -20
Average human: 0
Best out-of-the-box AI: 20
Wisdom of crowds: 25
Best AI superforecaster: 30
Professional human superforecasters: 35
Various prediction markets: 25 - 50?
World’s best forecasting team, under optimal conditions: 40 - 509
Perfect score: 10010
How can we predict the logical limit on forecasting? Although we can’t do this convincingly, in the spirit of Cotra 2020, let’s set some “anchors” and see where they land us.
Anchor 1: What if best human → AI superforecaster was the same level of advance as dumb model → prediction market?
Reeves’ implicit point is that gains from seemingly-exciting new technologies are small (in his study, 3-6% of RMSE). What if the advantage of an AI over a human forecaster was of the same magnitude?
We can convert Reeves’ RMSE into something called a Brier Score, and then convert the Brier Score into the Metaculus Score above. When we do that, we find that AIs would max out at a Metaculus score of 56.
Anchor 2: What if best human → AI superforecaster was the same level of advance as professional superforecaster → Samotsvety?
In 2010, we probably would have guessed that 35 (“various professional superforecasters”) was near max possible forecasting performance.
Since then, more people have joined the forecasting world, allowing us to find superstars who perform better than the previous stars, and the community has produced some tools and algorithms to support their work. Samotsvety, a team of well-practiced superstars using good tools, consistently beats “ordinary” superforecasters. If AIs beat Samotsvety by the same amount Samotsvety beat the 2010-era SOTA, according to Samotsvety’s own (generous) accounting, then they would max out at a Metaculus score of 65.
Anchor 3: What if best human → AI superforecaster was the same level of advance as human → AI chessmaster?
AI first beat the human champion at chess in 1997. Since then, it’s continued improving, and is now far beyond any possibility of human catch-up.
This alone doesn’t help us much. We want to know how close to optimal chess play top humans are. If they’re 99% of the way, but AIs are 99.9% of the way, that still could look like AIs beating humans every time.
But we can translate this into an objective space by looking at handicap. When AI first beat humans, its lead was fragile; a one-pawn handicap was enough to restore human supremacy. Since then, it’s been gradually getting better, and is now up to a 2.4 pawn handicap. It’s been a bit slower in achieving the third pawn than in the first or second, but not too much slower, and it’s too soon to say with certainty that it’s starting to plateau.
So it seems top human play is at least 2.4 pawns (let’s round this off to 3 pawns, since we have no reason to think we’ve reached the plateau11) short of optimum. What would it mean if top human forecasters were “three pawns” short of logical limits?
(some of my draft readers argued that chess is an almost optimal domain for AI - it can self-play millions of times for training data - and forecasting is much harder. I agree, but I don’t think that matters here - we’re not talking about how practical it is to create superforecaster AIs, we’re talking about what chess can teach us about the degree to which top humans approach optimal play.)
A three pawn handicap in chess is about 3x the difference between the world champion and a average grandmaster. If we round that off to 3x the difference between Samotsvety and Metaculus Pro Forecasters, the AI tops out around 85.
So What?
We sure have made a lot of claims about Metaculus Peer Forecasting scores, a number that approximately nobody cares about. How much real-life performance gain should we expect from max-possible AIs?
Consider a prediction market, currently at 50%, dominated by players somewhere between Metaculus Pro Forecaster-level skill and Samotsvety-level skill. And suppose that the market was about an event which would, in fact, happen. How confident would each of these max-possible-AIs, on average, be on the market?
Score 56: 54%
Score 65: 56%
Score 85: 62%
So these max-possible AIs would improve the accuracy of prediction markets by 4 - 12 percentage points.
This might sound disappointing - moving from 50% chance to 54% chance is hardly Nostradamus-level prescience. But I would actually be very excited by this. When I check something I care about on Polymarket, I usually have a decent sense of how likely it is. I’m not sure whether there’s a 55% chance or a 75% chance of Anthropic being worth more than OpenAI next year, but I know it’s not 10% or 90%. I visit Polymarket to see whether it’s more like 55% or 75%. On that metric, improving by 4% - let alone 12% - is actually pretty exciting.
I find these numbers more interesting than the worn-out debate over whether AIs or humans have reached the “max” and whether “perfect forecasting is impossible in principle”. It should be uncontroversial that the max is somewhere above current levels, even if only a fraction of a percent (for example, because even the best human forecaster sometimes has a bad day). And it should be uncontroversial that even the best possible forecaster can’t get 100% on everything (because you could ask questions about quantum effects that are unpredictable even in principle). Now that we’ve demonstrated that the answer will be some particular number, we can get to debating what number it is. If AI gives gains similar to past technologies, it could be between 4-12 %pp on prediction markets. Of course, if AI is much worse or better than past technologies, it could be some totally different amount.
Most people don’t have an intuitive sense of what 4-12 %pp on prediction markets buys you. I admit I sometimes blend together how confident I should be in an estimate by a professional superforecaster and how confident I should be in an estimate by Samotsvety, even though that’s a similarly-large difference. Probably all of this pales into insignificance compared to the gains we could get by switching from our current strategy of making decisions based on vibes and ballroom-related-bribery to listening to markets and forecasters at all12.
And that 4-12 %pp is only the headline number. A forecaster who is that much better at prediction markets would also be better at crafting good policy, or at identifying unknown unknowns (eg spotting the risk of a coronvirus-style pandemic even if this was otherwise so far off the radar that nobody had thought to write a prediction market question about it). Smarter important-question-identifiers and smarter forecasters together would provide a joint advantage greater than the benefit of either alone, and smarter policy-crafters could compound on this by turning that knowledge into useful action.
All of this keeps me excited about AI superforecasters, even though I don’t expect miracles.
Thanks to Daniel for looking over a draft of this post and making helpful suggestions.
This experiment would be illegal to conduct today.
Reeves et al argue against this interpretation in their paper:
That the poor discrimination of the baseline model carries so small a penalty in terms of RMSE is due in part to specific features of the NFL (e.g., salary caps) that ensure that most games are played between closely matched teams, and hence are decided with probabilities close to 50%. In other words, although the baseline model does perform poorly for high and low probability events, the relative rarity of such events means that it is not penalized much for these failures. One might therefore suspect that in domains (such as policy analysis) where events are not designed to be coin tosses, and where possibly the predictions of greatest interest may be for extreme probability events, less discriminating methods would perform correspondingly worse than they do here. To test this idea, we recomputed the RMSE of the six prediction methods exclusively for lopsided pairings between “winning” teams (that have won at least 9 of their past 16 games) and “losing” teams (lost at least 9 of 16). This subset comprises 37% of the data. As expected, the baseline model performed worse on these games (RMSE increased from 14.5 to 15.1 points), but RMSE of the TradeSports and Vegas markets, Probability Sports, and the win-loss model were approximately unchanged, at 13.0, 13.2, 13.4, and 13.5, respectively. Thus, even for these more extreme events, we find prediction markets again have only a small advantage over conventional forecasting methods.
I’m not really convinced, because I think the difference between the geopolitical data discussed later in this article and Reeves’ sports data does genuinely show that the sports forecasting must be harder.
Reeves paper argues that the prediction markets of 2010 were little better than simple models. But prediction markets have grown a lot since then. I asked Claude Code to replicate the study using modern sports bets on Polymarket and Kalshi. It claimed to find almost exactly the same results Reeves did, suggesting no progress even with orders of magnitude greater prediction market volume. I wasn’t able to follow its reasoning and can’t vouch for it.
There would be boring exceptions - for example, if an asteroid struck Earth and wiped out the AI industry.
Which is an obvious conflict of interest, but he’s working from publicly available information and I trust him. Thanks to Dan for looking over a draft of this post and making helpful suggestions.
The graph is logarithmic transform, meaning that going from -20 to 0 represents more genuine forecasting progress than going from 0 to +20. In theory this removes exactly the signal we’re looking for. But most of the transformation happens below zero, and we won’t be talking about that region, so we’ll nervously keep using this system rather than translate everything to some other space.
And the napkin math was increasingly reliant on AI inputs. There’s a slight risk this whole post is downstream of AI psychosis; I asked some smart people to proofread it, but probably they just asked their own AIs.
The statistics on Samotsvety’s Track Record page suggest 50, but the Track Record page is something of an advertisement, and although they report objective numbers, they have some leeway about which numbers to report. A reviewer suggested that in messier real-world conditions, the number would be more like 40.
By coincidence and generous rounding, not definition.
Fable seems extremely convinced that it’s impossible for any AI to ever play chess so well that it could win with a queen (=8 pawn) handicap, unless you could somehow encode adversarial-examples-relative-to-the-human-brain onto a chessboard. It thinks that optimum play probably plateaus at a 3-5 pawn advantage over human grandmasters. Fable wasn’t able to explain this to me in a way that made sense, but I know nothing about chess, so my incomprehension shouldn’t be taken too seriously.
A reviewer with minimal inside information but generally good judgment suggests that AI companies are already making strategic decisions based on advice from unreleased internal models, which is somewhat terrifying.