At least our opinion polls have better calibrated probability estimates!
But you are correct, though the issue is a bit subtler. For example, Metaculus advertises its track record: in past Metaculus aggregated predictions have been quite good, so it is worthwhile to look at their current predictions, too. However, they don't guarantee that the predictor pool stays stable nor even of equivalently high quality. Their platform is vulnerable to Eternal September.
The other part of answer is: do not underestimate power of the opinion polls. They are very useful, and they are not cheap to conduct either!
The true public opinion is valuable information which is difficult to obtain and access to it was restricted only to those who have the means to obtain it. And many people have an incentive to manipulate people's perception of what it is. How would you know what is the true aggregated common public opinion if you had only old-fashioned media available? You could talk to people you know, but that is only your social bubble, it could be biased. Suppose you watch TV and read newspapers. A reporter goes out and interviews the proverbial "man on the street" -- they are likely to interview several, and make a decision which interview gets airtime or is printed. Or talk show host invites several pundits to discuss what is the expert consensus on some phenomena or another -- they have editorial discretion over who they invite. And so on. You could call Gallup and ask for their rates -- not something everyone can do.
Regarding the degradation of Zelenskyy NO on Polymarket, a lot of these markets behave like options: they have a time value, and in absence of any new meaningful information, the time value falls until it reaches 0.
Good point! Given this time-value-of-options model, does it surprise you that the Zelenskyy NO price fell from 63% to 20% in the first 14 out of the 53 days until expiration?
It does not. I was a decently sized player in helping the odds go from ~60% to 20 or so. The bulk of the move was when market participants realized how inefficient Russian troop movements have been in Northern Ukraine.
Furthermore, the bulk of the shock move was done by one participant (a whale) who's trades move polymarkets substantially.
I see. Sounds like "market participants realized how inefficient Russian troop movements have been" was a bigger factor than "options generally go to 0 due to time value" in this case -- does that seem right to you in explaining what happened in the market?
Appropriately, "Will Kyiv fall by April" has dropped by more than half since March 7 (one week ago), whereas "Will Kyiv fall by June" has fallen only a little since then.
For a more colloquial explanation think of a question which goes "Will [Biden] [be president] by [December 31st]?" Particularly this works for questions when the answer is yes for today and you're asking if it'll change to no.
Every day that passes the odds should go down right? So looking back we could say (as Scott does), "this is decreasing linearly, better frontrun it". However this would be a mistake. More likely than small independent events which 'boost' Biden still being president (for Kyiv this would be small events which are pro-Ukraine) it is the lack of a large event which boosts Russia.
We can also think about a lottery. "Will I win the lottery before x date buying a ticket every day?" This would go down over time and is a much easier to understand and simpler model.
It's dangerous to immediately price in the updates you expect to see based on the trend! The forecasters are behaving as proper Bayesian agents and updating slowly, according to the trickle of evidence. Moreover, waiting makes existing evidence stronger - nobody debunked it, you've seen nothing that contradicts it despite time in which such information could surface.
"This is almost monotonically decreasing. Every day it’s lower than the day before. How suspicious should we be of this? If there were a stock that decreased every day for twenty days, we’d be surprised that investors were constantly overestimating it. At some point on day 10, someone should think “looks like this keeps declining, maybe I should short it”, and that would halt its decline. In efficient markets, there should never be predictable patterns!"
If you plotted a prediction for "will this iodine-131 nucleus have decayed by April 1" you'd also get a roughly linear decline (unless it decayed in which case it would jump up to 100%). Prediction markets are allowed to have "story arcs", so long as the *expected* change is zero.
My father's example was the Peso market. For a long time, investing in Pesos gave you a higher return than investing in dollars, which looks like a failure of efficient markets. Then Mexico devalued the Peso.
Similarly here. The pattern is what you would expect if, every day, there was a high probability of a small piece of evidence in favor of Ukraine and a very low probability of a very large piece of evidence in favor of Russia.
I am not a physicist, but IIRC the readioactive decay is exponentially distributed, i.e. "memory-less". It is of course true then, that you would expect this pattern for the iodine-131, but for a different reason than in the case of the war. The process of iodine-131 decay is Markovian because of its memorylessness (is that a word?) but the process of "who is winning the war in Ukraine" is not (the current state of the war is not enough to tell you less than knowing also the history).
But I was convinced by David's example with the peso and his translation to the war in Ukraine. I think it is a good description of how the market probably behaves and it kind of simplifies the situation to a Markov process where you have two absorbing states (Russia winning vs Ukraine winning in the sense of keeping its independence and Zelensky surviving). The process started somewhere in the middle (closer to the Russian absorbing state...pun intended) and has been moving to the other state since.
With a very high probability the process moves incrementally in the direction of the Ukrainian state and with a low probability it can move abrubtly in the Russian direction (from each state it can move directly to the Russian absorbing state or at least close to it, whereas it can only move incrementally in the opposite direction). The closer it gets to the Ukrainian state, the lower the probability of moving to the Russian state (but it can still jump there quickly if the trainsition does occur).
I guess this model makes sense. It seems that Ukraine is ready to fight for quite a long time provided that Russia does not do something very dramatic to force capitulation (capturing Kiev or something...I am not even sure that killing Zelensky would stop the Ukrainian determination at this point). Slow consistent Russian gains seem less likely, they have loads of problems with logistics, low morale, mounting problems at home (I don't think they would be willing or effectively able to fight until June, incidentally). So with that in mind, the markov model above makes sense.
I just wonder whether my description actually adds anything atop of what David said or whether it is not simply the same thing described in a more complicated way :)
What makes you say the progress of a war is necessarily non-Markovian? I would say good arguments can be given for it being a pretty Markovian process. That doesn't at all mean it's memoryless in the sense that anything at all can happen now, it just means its trajectory is determined mostly by what happened recently (with perhaps some constant bias in one direction or another). That seems rather likely to describe the course of a war, in the sense that what Belligerent A or B does at the start has a big influence on what happens next, and what happens next (and the next set of decisions) have a big influence on the next step, and so forth -- classic Markovian model.
A strongly non-Markovian model would be something a like a self-avoiding random walk, a situation where certain decisions made very early in the conflict have profound and somewhat random influence on things that happen much later. Like a bomb planted at time X that goes off at some unpredictable time X + dX (dX >> 0) and which then has a decisive effect. It's difficult to see how this could readily manifest in human affairs. What would this be? The chance of foreign intervention? Death of some critical leader? It would seem to presuppose a pretty contingent view of the evolution of a war, that "surprising" things out of the distant past could have unpredictably decisive effects on the present -- "unpredictable" in the sense that we could not easily trace their influence through time, A at time t1 caused B at time t2 which caused C at time t3, et cetera, *because* such a chain of influence is exactly what a Markovian process represents, a *non-Markovian* process is one where there are weird "Star Trek transporter" connections by which something far in the past leaps over all intervening events and affects the present all by itself, without going through any causal chain.
I think the discussion of Markovian vs non-Markovian is a bit of a non sequitur. I believe that the relevant property here is being a martingale not a Markov process. Obviously for continuous time (continuous) martingales there is not going to be any period of monotonicity almost surely, but in the case of discrete time, it is easy to see that for any delta > 0 there are martingales such that with probability at least 1 - delta the process is monotonically increasing and remains bounded between 0 and 1.
The martingale property comes immediately from mathematical finance, where a fundamental theorem says that no-arbitrage implies the existence of a measure under which the price is a martingale. Even ignoring the finance part, though, it should be obvious that we expect a (Levy) martingale. If we want the ERM predictor of the indicator of an event (with squared loss) with some information available in a sub-sigma algebra, we just take the conditional expectation. Thus at each time t, we should predict the expectation of the final event conditioned on the information available up to time t. This is just a Levy martingale. Where does the Markovian property enter?
Hmm, I guess you are right. The conditional probability should be a martingale. I guess this is the crucial bit that explains the behaviour.
I was thinking that a harmonic function of a Markov process is a martingale and the probability of ending in an absorption state of the markov process is a harmonic function.
I guess that Carl is right though that the Markov property is not that unreasonable here. What we really have at time t is not just the current state of the war but also all the previous information as well. And the bet is made based on this full information so it should trivially be Markovian.
I do not think it needs to be Markov; there could certainly be path-dependent information and my earlier comment still applies. Even absent the relevance question (I still do not think that it matters one way or the other for this question on the behavior of the process) I do not think that it is obvious that it even should be Markov, even in an efficient market.
I agree that it does not need to be Markov. My comment is rather justifying the model in case it is Markov, but you are correct that it applies more broadly.
You are saying that the reason is A whereas I was saying that it is because of B (and B implies A). You are correct that A is the important thing and there are ways you can arrive at A other than B.
Still, B still seems like quite a reasonable model to me after reflection (after thinking about Carl's comment).
But maybe I am missing a more elegant model which could describe the war that you might have had in mind?
Being a martingale is not synonymous with being Markov either in discrete or continuous time. There are plenty of examples one way or the other.
To give two trivial examples in discrete time, consider a process where X_0 has support of size at least 3 and y_n are independent Rademacher random variables for each n. Let X_n = X_{n-1} + y_n X_0. This is clearly a martingale. It is not Markov though because conditioning on the path (knowing X_0) concentrates the distribution for X_n given the history on two points while not knowing X_0 cannot have this concentration. (It is not hard to extend this example to continuous time by solving an SDE).
For the other way, take any Markov process (X_t) whose distributions conditional on the past are nonatomic and any function f that is subharmonic but not harmonic on the support of the X_t. Then the process given by f(X_t) is going to be a submartingale but not a martingale. This applies both in discrete and continuous time.
Ah I see what you mean. Thanks for the example. It's pretty contrived, however. I think for the real-world processes we are discussing here the distinction is not relevant.
Markov processes and martingales have a close relationship (via harmonic functions, also solutions of "well-behaved" SDEs are Markov processes) but they are indeed not synonymous and neither necessarily implies the other as Adam mentions.
On the other hand a lot of simple examples of martingales are markovian and vice versa (or their functions are, such as the relevant probability above) and a "markov process of the war" I am describing above is still the most intuitive toy model for me.
Exactly! Basically the pattern speaks to how little we know at a "gears level" about how the war is going. Given the right prior distribution I'd expect the prediction to evolve the same way if our only source of new information was a webcam in downtown Kyiv.
It's worse than the iodine-131 example; it's more like "will this iodine nucleus have decayed by April 1" when you don't even know what isotope of iodine you're dealing with.
I'm kind of OK with Metaculus's initial and current numbers, but the linear-ish trend seems off. The first week gave us a *lot* of useful information. First, it told us that the Ukrainian army wasn't going to fold like the Afghan National Army, that the people would back them up, and that Volodymyr Zelenskyy had the right stuff. All of those were in real doubt on 24 February. Second, it told us that this is not your father's Guards Motor Rifle Division we're dealing with, that the Russian Army of 2022 is not the one we were worried was going to overrun the Fulda Gap in 1982 but has severe logistical, organizational, technical, and morale problems. We *sort* of knew that on 24 Feb, but not with high confidence.
So most of the fall-off should I think have occurred in the first week, ruling out most of the "Shock and Awe prevails" scenarios. Rather like, if I hand you one of the seven major isotopes of iodine picked at random, there's a good chance that it's one of the ones with a half-life of hours to a few days, definitely not going to last a month. But a week later, you pretty much know that you're dealing with one of the longer-lived isotopes and maybe the actually-stable one.
Now I want to spreadsheet up a forecast of the decay of a random iodine atom in one month, conditional on it having lasted N days already.
Unless I'm mistaken, you can easily compute this by hand: Let $T$ be April 1, $t$ be the time of the first occurrence of decay, $p_s=\mathbb P(t\leq T\mid t>s)$ be the probability of decay by time $T$, conditional on no decay up to time $s<T$, $A_i$ be the event that we're looking at an iodine nucleus with half-life $a_i$ for $i=1,\dots,n$.
Then we have $p_s = \sum_{i=1}^n\mathbb P(A_i\mid t>s)\mathbb P(\text{Exp}(a_i)\leq T-s) = \frac{\sum_{i=1}^n e^{-{s\over a_i}}(1-e^{-{T-s\over a_i}})}{\sum_{i=1}^n e^{-{s\over a_i}}}$.
Intuitively, due to the exponentially decaying tails of exponential distributions, once we observe no decay after 2h we can almost certainly rule out half-times of <1h (provided there is an alternative hypothesis, like the possibility of another particle with half-time >2h). Once the contribution from a particle with longer half-time becomes dominant, linear decay seems reasonable.
For the record, I was primarily interested in how much this particular toy model would deviate from a linear function and would not use this for actual predictions.
(Apologies for the lack of formatting, I couldn't find a way to properly display links/equations on here.)
to answer the question, no i don't think that drop is strange. over that two weeks, we saw the results and new information adjusted quickly, then it should slowly decline as the deadline approaches.
Sorry, I'm probably just reading the graph badly, but what period in 2/28 are you thinking of as the "adjusted quickly" period, and what period is the "slowly decline" period? (The graph that Scott posted of the metaculus aggregation over time seems to move pretty smoothly from 2/28 to now; I don't see an obvious "adjusted quickly" period.)
The way I interpret and see the chart, it had some relatively old value Feb 28-Mar5, then steep drop to new lower value, then slow decay from there. This may not be the case, but the main point of my comment is to describe the behavior after that Mar5 drop. The comment from House Always Wins illustrates that point perhaps more articulately.
Ah, thinking of 3/5-3/6 as being a quick adjustment, and the resulting slide from 40% to 14% in the 8 days of the remaining 26 seems helpful. I still wouldn't have guessed that 2/3 of the probability mass "should" lie in the first 1/3 of days, so it does seems like there's more information being incorporated than just "probability decreases with each day closer to the April deadline that Kiev hasn't fallen".
The slowness of Russia in the early phases of seizing Kyiv suggests a slowness of their attack in the later phases. The distribution of "how long it takes Russia to get its troops to Kyiv," "how long does it take Russia to start shelling Kyiv with artillery," "how long does it take Ukrainian resistance to break down," "how long does it takes to seize the first raion," "how long does it take to seize the second raion," etc, are not going to be uncorrelated, even if they don't all have the same distribution.
A prediction market for next week’s Powerball numbers makes no sense as that is an unknowable thing. Similarly how Putin, Zelinskyy, Biden, Sholtz, random software problems that accidentally send a Russian Cruise Missile to Warsaw all interact to influence the future is just as unknowable as the future state of the balls bouncing around at lottery HG.
A prediction market for next week's Powerball numbers should have every combination priced at around ~0.000000342229781%. The balls bouncing around have a huge number of unknowable interactions, so it's hopeless to form predictions based on predicting every step of the process. But, like with the Powerball *results*, it's possible to estimate what the distribution of *outcomes* might be, even if it's hopeless to predict every footfall of every soldier.
It isn't *just as* unknowable as Powerball, even if it's quite unknowable. All those men you named have stated their intentions and have a history of behaviors in situations of varying degrees of similarity. Human behavior can be predictable to some extent. Degrees of confidence exist.
That’s true if you’re predicting the outcome of general events. The life insurance market works great at predicting how likely you are to die in the next 10 or 20 years. It’s predicting specific events that I’m not sure about.
For example - Tesla wants to take out $100 billion in life insurance to protect itself from Elon commuting suicide or other mwise ending his life prematurely. How much value is the life expectancy of the set of all 50 year old males in the US?
It provides a good base rate, which you can adjust upwards or downwards based on Elon Musk's particular characteristics (e.g. is he depressed, does he drink and drive, does he do dangerous drugs, etc).
Superforecasting, by Philip E. Tetlock, covers the specifics of how those who make these kind of "one-off" predictions accurately manage to do so. Unfortunately, it's sufficiently long and detailed in its coverage that there are few money quotes, though somebody reproduced the "Ten Commandments" from the appendix here: https://www.lesswrong.com/posts/dvYeSKDRd68GcrWoe/ten-commandments-for-aspiring-superforecasters
Define "Musk-like scenarios"? If you mean scenarios which rely on complex, multivariable human decision making similar to the ongoing war in Ukraine, you can just go to https://www.metaculus.com/questions/track-record/ and look at Geopolitics; Metaculus gets significantly above monkey levels, though I don't know how it compares to braindead heuristics like "the next five years will be identical to the previous five years".
If you mean literally predicting when specific people die, that's not a very common thing to see on prediction markets, probably because of the moral hazard.
A prediction market for next week's Powerball numbers is perfectly legitimate - the correct probability to assign to any number would be 1 / 1 million (or whatever the actual Powerball odds are). The only reason people wouldn't use such a market is that it would be so obvious that nobody would bet against the 1 / 1 million number.
What's the correct odds (ie the equivalent of saying "1 / 1 million") for Russia taking Kyiv this month? I'm not sure, but it's probably a different number than the correct odds for Russia taking Mariupol this month, and it's fair to speculate on both sets of odds.
The fact that we can't get certainty is fine, all prediction markets are asking you for is a probability. You probably have opinions on this probability already - 99% would seem unreasonably high, 0.1% unreasonably low.
People definitionally bet on Powerball numbers at worse than their actual odds of winning. So in principle there should be people available to bet that the odds of a given combo are higher than they actually are based on gamblers' fallacies or magical thinking.
No one is saying that you could use a prediction market to predict the exact winning numbers. They are saying that the prediction market will give an accurate PROBABILITY for a given set of numbers.
That is, if you ask a prediction market "will all 20 of these dice come up as 6s?" the market will say "with probability 1/6^20", which is the best possible answer given the available information.
To a hypothetical person who doesn't know how to calculate that probability for themself, that would be useful information.
That hypothetical person would have to be significantly dumber than average, since the market merely presents the conventional wisdom. So the question might be: why would a prediction market be useful to someone *other than* the unusually clueless?
I mean, the stock market tells me what the conventional wisdom is on the future earnings of Tesla, and that would be useful if I had never heard of the company. Gee, maybe buying Tesla stock makes more sense than putting my money under the mattress.
But if that does not describe me -- if I know as much as the average schmo -- what useful information can I extract from the present price of TSLA? Pretty much zip. If I buy when the price is high, and sell when the price is low -- i.e. I follow the conventional wisdom the market summarizes -- I will never make money. The *only* way the information can be practically useful is if I have *more* information than the average schmo, e.g. I am insider (or just smarter than everybody else) and I happen to know the conventional wisdom greatly under- or overvalues Tesla. In that case I can use the current market value to know when to bet against the market and win.
I guess I'll add that it is nearly definitionally not possible for my extra insight to dominate the market, so that the market reflects my extra knowledge. The market isn't exactly a one-man-one-vote democracy, but it's also not super duper different from that. Except in the rare circumstances that I am staggeringly rich *as well as* unusually well informed, my influence on the market (even as I make a killing) is not going to be enough to shift the conventional wisdom.
Obviously, the information from the market won't help you **at beating the market**.
If you know something the market doesn't, you can make money, and in the process make the market more informed. If the market knows something you don't, then you can learn something, and use that information ELSEWHERE in your life.
You might even be able to do both of those things on a single question, if you and the market know *different* information. ("I don't know how the market decided on a price of X, but I have inside information I'm sure the market hasn't taken into account, so I can guess the *direction* of its error.")
But for any given bit of info, you can either learn it from the market OR use it to make money from the market, not both. Markets are not perpetual motion machines.
(And even if you were going to try to use the market's information to play the market, "buy high, sell low" would be a profound misunderstanding of what the market is telling you. If you had an option to buy or sell at a fixed price of $X (regardless of what price the market is currently trading at), then looking at the market price could tell you whether you should exercise that fixed-price option. But if you're talking about buying or selling **at the market price**, then a high price is NOT a recommendation to buy--it indicates the value is high, but THE COST IS ALSO HIGH, so they cancel out!)
Sure, but those things are only true pretty far out on the statistical tail. I have to know *a lot* more than the market, because I need to know enough to know I'm better informed than the market. Given the epistemic bias of the rational actor against thinking he knows something special -- he's dead right and millions of others are dead wrong -- it would take a very well informed person to rationally conclude he can safely bet against the market. Similarily, it takes an unusually poorly-informed person to realize the market knows much more than he does, since the market doesn't have any special qualifications -- it's made up of average people. How badly informed do I have to be on any subject to conclude reliably that the averae schmo on the street knows much more than me? Pretty badly.
So in both cases *as a prediction tool* the market is only useful for a small demographic at either extreme. That wouldn't justify its existence, since it would be ipso facto useless for the great bulk of its participants. Why would anyone willingly participate in a market the main positive effect of which is to make unusual people who are not you better off? Nobody's that altruistic.
People *do* participate in "prediction markets," e.g. every time they go to the track and place a bet on the horses. The existence of the odds -- or indeed the market in general -- benefits only the people who know nothing at all about horses, who can avoid putting their money on the 100-1 long-shot, and the people who know *everything* about horses, who can *put* their money on the 100-1 long shot that they happen to know is going to win for some obscure reason or other. The rest of the people participate not because they seriously expect to come out ahead, financially (unless they're idiots), but because people generally get a thrill out of betting on stuff for some weird psychological reason. (Perhaps because by strategic forgetting of our losses we can feel smart about our wins.)
Yes, I agree, if I have a fixed option the market can tell me whether to exercise it. Under what consideration could I have something like that in a prediction market? I am not following how this would work in, say, a prediction market on the Ukrainian war. How could I come into possession of such an option, in a natural way? If you have a specific example in mind, it would help me understand what you mean.
Couldn't this market effective simulate a decentralized lottery? Instead of buying an actual $2 powerball ticket with a 1/1 million chance to win a million dollars, someone buys a $2 share on a specific number. Enough people enjoy the tiny chance of winning a huge amount of money that they're willing to do this for real lottery tickets even with negative expected value. Then all it needs is a bunch of rational actors with enough money to provide liquidity in exchange for profiting from the irrational choices of the gamblers, who would probably end up running automated programs that arbitrage any discrepancy between the current market and the mathematical odds. The total amount of money in the system would vary, but in expectation the return for winning should be approximately the reciprocal of the odds for winning, minus any losses from irrational behavior, which are profited by the arbiters.
This seems like it would end up remarkably similar to a regular lottery, except instead of the lottery provider making tons of money from the expected value discrepancy, multiple potential liquidity providers would compete, driving the profit margins down and creating a lottery that was more fair towards the gamblers.
Depends on whether the purpose is to have fun gambling, or actually predict stuff in a reliable way. If the latter, then the necessary underlying assumption is that the future is already determined -- the die is already cast, and it's just a question of discerning which side will face up before it actually happens, like predicting the path of a thrown rock in a vacuum from its initial position and velocity.
We can fudge it a little bit by saying it has to be predictable only in a statistical sense, like ordinary statistical mechanics, or quantum mechanics, e.g. whether *this* time we get nuclear war is unknowable, but we can say with certainty that the odds are 2.633%. That doesn't really change anything, because the predictability still requires perfectly deterministic underlying dynamics -- no chaos, no random events.
Is it possible that 'story arcs' in prediction markets come because most people have far heavier priors on other people's assessment and judgement, than they do on events in reality itself?
Maybe most people, even those using prediction markets, have a real low prior on something like 'prediction markets are totally wrong here' ? I have far more experience with 'prediction markets being general correct' than i do with forecasting military outcomes.
Do we assume that the mother is already taking prenatal vitamins before these recommendations are given? My guess is yes or else folate/iodine/etc will top the list. This matters for things like vitamin D (already in the prenatal pill, so I guess you mean extra D on top of that) and the vitamin A variants (vitamin A is in the prenatal pill, and too much of it is actively harmful, which I assume is why you list these specific lutein/zeaxanthin variants instead of a generic A pill).
"One possibility is that, by a crazy coincidence, every day some new independent event happened that thwarted Russia and made Ukraine’s chances look better. Twenty dice rolls in a row came up natural 20s for Ukraine. Seems unlikely."
That's more or less exactly what's happened. It's not unlikely at all, it's how we experience the world.
On day 1, Ukraine was still standing.
On day 2, Ukraine was still standing.
On day 3, Ukraine was still standing.
etc etc etc
These are all new information! On any given day N, different things might happen, we don't know. We have to wait and observe. Simply drawing a linear extrapolation through the first few days would be lunacy.
Also, labeling these as "natural 20s" is a huge overstatement of what we know about the underlying distribution. We have very little information on which to base an estimate of "Can Ukraine defend against an attack by the Russian army?", and I offer as evidence the fact that this post exists, we're having this conversation, etc etc. So it *may* be that Ukraine is on a hot streak and rolling 20s, or it may be that the outcomes we see are right in the middle of the distribution because it happens that Ukraine is a better match for Russia than some (many? most? idk!) people thought.
I agree, perhaps the market is approximating a “no info environment”. Suppose I arrange to watch a movie at 3pm with a friend. My friend tells me he will try to get there on the early side, but when I arrive he isn’t there. Now how does the probability of him arriving in time for the movie update every minute? I know literally nothing about his progress save the passage of time. I think it looks quite a bit like the Kiev market.
I was thinking about this problem recently while writing code for estimating the speed of a spinning wheel from a pulse train. It's tricky even if you have an estimate from previous pulses.
Consider a bus stop, where expect the bus to arrive every 10 minutes. Up until 10 minutes, you can estimate time remaining based on your original estimate. But now the bus is late, so what do you do?
One way to do it is to assume the bus is always about to arrive. At 11 minutes your new estimate is that buses arrive every 11 minutes. And so on. When the bus finally arrives, your prediction will be correct.
This seems to result in a smoother graph when the speed of the spinning wheel is slowing down. Adding elapsed time to the estimated period results in the frequency decaying like 1/x.
Sometimes, though, the bus will arrive early, and then you need to revise your estimate up.
This is sort of like saying the Lindy effect applies Ukrainian resistance.
But what bounds can we put on this? I would expect that cities under siege (completely surrounded) can't hold out forever. On the other hand, people say the Russian Army has logistical limits on how long they can continue the invasion.
Who has time on their side? The theory of protracted war is apparently that the Ukrainians win by surviving. Perhaps the markets believe this? Or maybe that's just the structure of this bet, since it has a deadline.
Kiev is not completely surrounded, so that changes your calculus a bit. The Russians have not been able to cut off access to the city from the south, and haven't made much progress in that direction over the past few days at least.
I'm thinking that one of the worst things about surviving a nuclear war would be finding yourself in a society organized and dominated by the kind of people who optimize their lives around surviving a nuclear war.
The vast majority of survivors would be people who survived by some combination of coincidence and general competence, no matter how (if) it happens the vast majority of survivors will not be nuclear preppers.
"Preppers are horrible people even if they're right" is enough of a trope in post-apocalyptic fiction that I've become suspicious of it. It reads like status gaming by the big winners of current society (which produces very different values and constraints that they've in turn optimized for) against their outgroup. Selfish, suspicious anti-authority types with unpopular beliefs are exactly the people I'd want to side with in a post-apocalyptic environment.
Preppers are people who are irrational, but happened to be right by pure coincidence. The fact that they are right doesn't change the likelihood of them being irrational.
If we were invaded by aliens who could be stopped by wearing literal tin foil hats, I'd expect the survivors to be even worse.
This past month has really brought back the '80s, when "is nuclear armageddon more or less likely this week?" was a perfectly normal topic for news discussion.
> If there were a stock that decreased every day for twenty days, we’d be surprised that investors were constantly overestimating it. At some point on day 10, someone should think “looks like this keeps declining, maybe I should short it”, and that would halt its decline. In efficient markets, there should never be predictable patterns!
Most high-growth tech stocks have been in secular decline since November (see eg NFLX, COIN, DASH, PTON, U, etc), so if you really believe this, you should go pick up those obvious $100 bills just laying on the ground :)
Ha ha yes that's why I always buy stock when the price is increasing and sell when the price is decreasing, which makes perfect sense except gosh darn it I seem to have less and less money every year...
No no. When price is decreasing, you don't sell (you are already too late to cash in). When price is increasing, you don't sell (you are too early to cash in).
Question from the audience? Let me repeat that everyone hears it. Gentleman here in the studio asked, "When do you buy"? The answer is, whenever you have sufficient funds, or at random intervals, sir.
This is the strategy that allowed us, BUY'N'HODL Inc, to finance our fancy interstellar home, the Axiom. We will return to planet Earth after it has been sufficiently cleaned.
Another explanation for the slow decline in Ukraine predictions:
This might happen if people are thinking something along the lines of "Russia will not capture Kiev by April unless they do something to turn things around before then". If so, then every day that Russia fails to turn things around provides a bit more evidence that Russia will not capture Kiev. This doesn't violate conservation of expected evidence or the efficient market hypothesis, because every day there is some small chance that Russia WILL turn things around and the odds will swing substantially in the other direction.
I'm kind of echoing what's been said before in the comments, but I'm really not sure why the monotonically decreasing pattern is odd. A stock shouldn't decline over time, but a stock is also in theory an infinite prediction (there's no time expiry). A prediction with a specific deadline is quite different.
Suppose your mental model is something like "there is a 1% chance Kiev will fall on any given day". With 30 days until April, you'd predict about a 74% chance of Kiev not falling (.99 ^ 30). With 15 days left, conditional on it not falling yet, you're up to about an 86% chance. Your base rate probability of Kiev falling doesn't have to change (it's still 1% on any given day), but the decreasing time to expiry changes your odds of the actual contract resolving one way or the other.
I basically agree with this take. Every day Russia fails to make progress in taking Kyiv is a day closer to the deadline. They might still be making progress, but so far it's too slow for them to make an April 1st deadline. I think their best bet is in surrounding the city and starving out the defenders, but that'll take a couple of weeks even after they've surrounded the city, which they're still far from doing.
I think that the other thing is that people have been expecting the Russian army to figure out its organizational problems but they haven't done that yet. The longer they fail to do that, the more it looks like they won't be able to do it. Which means the war will be continue to be slow-going and costly for them.
I think the key here is that the evidence has mostly been of the following form:
"the Russians haven't moved very far, and they've already been invading for X days"
That's evidence that becomes monotonically stronger as X increases but there's no sharp cutoff point where it qualitatively changes in character. It's quite different from the updates you would get from crucial pivotal events (agreeing to ship a more advanced SAM system, the fall of a city, losing a key skirmish etc) but I also think it's extremely relevant evidence.
For instance, if we think one of the two situations is likely to be a correct description of the situation, we could reasonably believe that every day without solid Russian progress is evidence towards option 2 and update accordingly.
Scenario 1) Russian forces are fundamentally powerful enough to take Ukraine but take time to deploy effectively
Scenario 2) Russian forces are not powerful enough to take Ukraine
Every day X ticks one higher, and the P(evidence| scenario 2) becomes higher and P(evidence|scenario 1) becomes lower meaning a larger update but incremental update towards scenario 2.
Even now, it's still possible that the Russians are gradually getting into position and will still prevail, but the alternative story where their logistics, morale, tactical awareness etc are too constraining to achieve their objectives has become a lot more plausible.
Yes, this was a weird objection to me. The monotonically decreasing with time in the absense of major new developments seemed the obviously correct reaction.
Zvi has very similar behavior in his Covid predictions. For example, before the recent Hong Kong surge, as time went on with no (apparent) changes in China's case load, he gradually incremented up the likelihood that they would successfully contain the disease. Then, something major happened (the Hong Kong spike and various smaller outbreaks) and he dramatically reduced the likelihood.
(This was a comment made by the real Enopoletus Harding on Metaculus.)
I thought of leaving here due to my disagreements with the moderation here, but since Scott has mentioned me on his blog (I can't comment there since I'm banned for extremist posts), I will continue to make predictions here. Needless to say, the chances of Russia taking Kiev go down each day there is no advance. I personally don't think it makes sense to update very rapidly, since, for all I know, the Russians may have been using the past two weeks to prepare some lightning assault -not something that's likely, certainly, but the chances of that are too high to simply exclude off the bat. Maybe I should update more rapidly -I've been underperforming the community when evaluated at all times. But knowing when to update and having good starting positions are not easy skills to learn.
I've also created an account on Manifold Markets, I expect my predictions to be better there (I have a habit of putting the vast majority of my money on a few markets I can easily win in):
No, I'm not trolling. Wrong, maybe. It's happened before. I had 10% chance of a Russian invasion for way too long, but I obviously changed that as the evidence became more clear.
Ukraine is going to lose in the end. Maybe only after most cities are destroyed, but they cannot win. Then they will be ruled by russian marionettes & dictat in the foreseeable future. The dictat will be harder and longer, in particular on the West-ukrainians, the more blood the Russians have to spill to conquer the country. The only alternative is for the ukrainians to cut their losses some way, but so far that seems unlikely. I'm writing this because the endgame is what matters. Making "predictions" about which week Kyiv and other cities fall is a teenager-way of thinking and acting about a real-life war. It is of very marginal importance which week it will happen, and such speculations are also rather distasteful. Grow up.
The big questions rely on and derive from the little questions. The most accurate way to make large predictions is to break them down into many smaller subpredictions for which we can derive some reference class. Or, to borrow Jesus's phrasing, "Whoever can be trusted with very little can also be trusted with much."
The longer Kyiv holds out, the more time there is for some major event to change the entire equation. A general collapse in one of the two armies, for example. Of for another major power to get directly involved, for another. The math might currently be in Russia's favor (actually it's not that clear; it depends on what the casualty numbers are really like,) but there's still time for major events to upset matters.
It's also important to understand that it's not just the "endgame" in Ukraine that matters. This war affects most of the world at the moment, and whether Ukraine continues to fight matters a lot to what the other powers involved should do about it. For example, if Ukraine can be expected to hold out for another few months or a year, then Putin needs to worry about supply and domestic issues so it affects him greatly. It also affects the kind of aid it makes sense for the EU and NATO to send to Ukraine. It also affects what countries that used to depend on Ukrainian grain will have to do to deal with their own crisises. So yes, this matters.
"Teenager"? It seems like you can make real money betting against the people you disagree with. There are lots of adults who making a living on speculative markets.
My mental model for the progress of this war is essentially as a race - damage to Ukrainian civilians and military may push Ukraine to concede a disadvantageous peace, and damage to Russian military and economy may push Russia to concede a disadvantageous peace; both of these damages are progressing at varying speeds, and the question is which one of these "progress bars" will reach a breaking point first.
And I strongly disagree with your assumption - I don't see how Russia can force *that* kind of peace even if their military operation completely succeeds now. If Russia conquers or destroys all the major cities and installs a puppet government, that does not enable Russia to rule over Ukraine - the populace will not agree to that, not even in the eastern Russian-speaking areas like Kharkiv, not after what has been done in the war. And Russia can not afford enough manpower to enforce proper control, that would require something like 500k permanent garrison there; so Russia will have to leave and their marionette regime falls right afterwards (as after USSR-Afghanistan). I mean, pacifying and controlling a hostile country is really, really hard; it has not gone well for USA recently despite immense resources, and even for the quite effective Stalin-NKVD apparatus it took many, many years after WW2 to suppress guerilla warfare in Eastern Europe - and those people and Taliban didn't have external support/arms supply from all the wealthy countries, like Ukrainian guerillas would have in case of a Russian conquest. So as Russia can't plausibly hold Ukraine, they would have to concede to something more reasonable (e.g. recognition of Crimea + DNR/LNR, perhaps a bit more land, but Ukraine government remaining in place) even in the case of an overwhelming victory - which itself is far from certain.
In my opinion the main part of the war has not started yet - the Ukrainian strategy apparently was to trade space for time, preparing for the actual fighting in taking over the large cities, which is going to be difficult for the attacker, and where the war would be decided. So far Russia is only assaulting the relatively small Mariupol (Kherson and Melitopol are many times smaller than that), not even trying to make serious progress in Kharkiv or Kyiv. We can see how much men and time it took Russia to take Grozny; it seems plausible to consider that Kyiv - which is much larger and has much more resourced defenders - will take at least as much.
Im skeptical of the "russia will never submit Ukraine". Didn't they do it with Chechnya pretty succesfully? Seems like they could pull it off at least on a portion of Ukraine.
Chechnya is much smaller, but Ukraine doesn't have much in the way of mountains.
Chechnya is a very relevant example of how they were very much *not* able to do it "pretty successfully". They tried to enforce submission for many years and failed, and they succeeded only with a combination of keeping excessive military garrison (at levels which Russia can't afford at the larger scale of Ukraine) plus effectively buying off the local leaders and population with excessive subsidies (again, at levels which Russia can't afford at the larger scale of Ukraine) and political concessions.
The Chechen resistance did not end with surrender and submission, the current result is "an understanding" where formally Kremlin gets a victory, territorial integrity and declarative submission, but in practice devolves severe autonomy and large monetary "tribute" to Chechnya. Kadyrov is not like other regional leaders which are at the mercy of Moscow, he is independently powerful, with his own army (that is not accountable to Russian hierarchy), and is effectively an ally of convenience to Putin personally and has a lot of de-facto autonomy from the rest of Russian authorities - there are all kinds of incidents with Kadyrov's people in violent conflict with other Russia region's court system and police system, and having de facto immunity for that. I don't have a specific English source in mind, but if you're interested about this aspect, I would recommend you to read about how Kadyrov came to power, all of that process was quite interesting.
I don't know any - it's relatively recent, most written information is in Russian with little analysis from the western part, and I doubt if the authorities would allow to publish an analytical book like that in Russia (and IIRC both threats and actual violence has been applied both to local Chechens and expats commenting on Kadyrovs regime). However, I just stumbled upon a quite extensive summary in English in Twitter-thread form - https://twitter.com/kamilkazani/status/1497612331953577991
I'm not sure if your claim is "I'm so obviously right that it's useless for people to try to predict things" or "the things you're predicting are too small scale to matter".
I disagree with the first because the people who say the first have a worse-than-average track record of being right, and I disagree with the second because big things are made up of small things. I bet there are a lot of Ukrainian generals who would like to know in what month Kiev will/won't fall right now, and would change a lot of what they're doing if they knew.
But also - realistically everything that's happening on prediction markets now is capacity-building / proof-of-concept. The more we establish a record for prediction markets on small things, the more likely we'll have them ready and established when we need to use them for big things.
(the Arb report I linked finds that forecasters beat epidemiologists and models in predicting the spread of COVID-19 and its effects on interventions - seems potentially useful)
I would like to go on record that I find that poster's attitude distasteful AND I disagree with his object-level prediction about the end-result of the Ukraine war.
However, upon reflection, he may be on to something, and I don't think your answer addresses that something. A more sophisticated, steelmanned version of the sentiment he expresses would go as follows:
It should be, of course, uncontroversial that every little data point brings us closer to a correct big picture view. However, it does not follow that a *prediction market* about every little single data point gives us more information. Realistically, prediction markets about little things will be extremely correlated with each other, influenced much more strongly by the overarching big picture sentiment looming above them than by any actual new information directly related to the question they're trying to answer. (And even if one of them is actually pricing in some important niche knowledge, we have no way of teasing that knowledge out from the single probability number that the prediction market produces.)
In other words, in a flow of information from primary data to their increasingly more processed derivatives, prediction markets about small things may actually be further down the line than the big picture view, with the big picture view having already absorbed the relevant information from the primary small-thing datapoints. The corollary to this would be that predictions about small things should not, and probably do not, reflexively influence the big picture view. This is an important, non-trivial point that, honestly, I find likely to be correct.
I'm actually trending towards thinking that Russia is not likely to win unless China props them up. At this point it's looking like a long, long grind and I don't know how long Russia can sustain that with the way their economy is going.
this is an example of definitions being really important - as the definition of 'win' is the key factor here. And where prediction markets on those various definitions of 'win' are highly valuable
Perhaps the best outcome for Russia is a 'loss' in Ukraine.
A Russian 'win' leaves Russia maintaining a very large military force in the country for years. Possibly decades. A never ending money and resource pit. This was true from the moment it become clear that Ukrainians were not hoping to become part of Russia.
Especially in a scenario where Russia trades their withdrawal for significant sanction relief. But even in a scenario without that, I think Russia withdrawing would give some portion of parties participating in sanctions the cover they need to slowly back away.
Why do you think they cannot win? Russia is about three and a half times the size of Ukraine in population, which is an advantage but may not be enough to balance the facts that Ukraine is on the defensive, that the Russian military appears to be very poor and Ukraine's military pretty good. Russia is richer than Ukraine, which in most contexts would be important, but Ukraine is being supplied by NATO countries that are much richer than Russia.
It's hard to be sure but it looks as though, despite its size, Russia has a very limited number of trained soldiers, many, possibly most, of whom have already been committed to the invasion. So I think one plausible outcome is that the Russian military ends up unable to advance further, possibly unable to hold what it is currently occupying. They can kill a substantial number of Ukrainians, but not a substantial fraction of the population, and at the cost of further losses of men and material.
"Ukraine is going to lose in the end. Maybe only after most cities are destroyed, but they cannot win."
Your confidence in the ability of the Russian army to sustain its morale and current operational tempo indefinitely, is I think unjustifiably high. The best unclassified sources I can find suggests that the Russian army has already lost 10-20% of its *total* effective combat strength in this campaign. That can be reconstituted, but not in real time. And Putin isn't going to run his army down to literally zero combat effectiveness.
The 30% of forecasts where Russia hasn't even taken three major cities by June, those are the ones where Ukraine probably wins. And, yeah, maybe promises to recognize Crimea as part of Russia to dissuade Puting from spite-nuking Kyiv, but that's still a win for Ukraine.
"Russia will probably win" is a reasonable forecast. "Ukraine *can't* win", is not.
> One possibility is that, by a crazy coincidence, every day some new independent event happened that thwarted Russia and made Ukraine’s chances look better.
This doesn't require Thursday. The "event" is just the absence of any big Russian breakthrough.
If Ukraine is stronger than anyone could observe, then each day of non-defeat reveals this fact just a little bit to a rational observer
I'm not sure I follow the statement "At some point on day 10, someone should think “looks like this keeps declining, maybe I should short it”, and that would halt its decline."
It doesn't make sense to me that prediction markets would improve estimates without additional causal information (information tied to actual "facts on the ground") vs. this notion of the market just predicting based on itself, which seems antithetical to the goals of meaningfully accurate predictions. Beyond the market forces at play (which are just a tool to motivate effortful predictions out of forecasters), "technical analysis" (as opposed to fundamental analysis) adds little to the causal information included in prediction markets' final estimates, correct (unless someone believes that the best forecasters are always leading major shifts/trends rather than lagging them, which seems questionable; why couldn't poor forecasters move the market?)?
This feels like a fundamental limitation of the current approach/emphasis on forecasting that I've never been comfortable with: trying to predict something important by staring at a stock ticker rather than looking at the facts on the ground (even given those facts are fuzzy/noisy and their interpretation is also fuzzy/noisy). Moore's Law held for the past 50 years. Therefore, based on that, many futurists (as far as I can tell) just extrapolate out that growth in hardware performance and cost indefinitely into the future rather than ask whether there are any concrete reasons to believe semiconductors can keep being shoved into dramatically smaller form factors (Moore's Law has failed to keep pace for about a decade as far as I understand). It feels like this idea of "the trend is moving this way, therefore it will continue" is fundamentally antithetical to the whole point of forecasting and serves just to reinforce current market predictions. "a real dog" points out that delayed/lagging Bayesian updating strategies are useful, stabilizing heuristics, which I'd agree with although this a form of arguing that we should be biased to towards our previous priors more than we would otherwise be (if we didn't assume that new info is inherently less trustworthy than older info). That said, this feels different from trying to predict new things (rather than old things that were presumably based on causal info) based on a trend. The Kyiv trend looks kinda linear; in 2 weeks it could go down to -50%!
In this case it seems like the prediction market forecast is too biased toward prior predictions given how rapidly the situation is changing, as Scott pointed out. However, it's a different thing to say "given the most recent news [causal information] I've seen, the current market is off and I should short" from "given the trend is moving downward, I should short", which assumes all changes in the market should be accompanied by positive feedback effects (more formally, in linear control systems, this is equivalent to trying to always have the feedback time constants perfectly tuned, which is hard, particularly in unpredictable situations and I'd imagine would lead to instability; how long is a "long trend" that we should predict based on vs. a "short one" that is just spurious noise? How do you know in advance?). The trend only looks clearly linear in retrospect... at least for me, not being an expert of course, the situation didn't feel so linear at the time.
On the nuclear forecasting I do wonder if one month is too short a time horizon for us to expect increased risk. In my mental model of escalation we would see several weeks or months of increased tensions that gradually drag NATO in and make nuclear exchange conceivable. Perhaps Russia bombs a school next week and the west responds by putting the MIGs on trucks and shipping them to Ukraine a week later. Perhaps a missile aimed at the trucks hits a Polish border station and NATO increases shipments. A week later after some of these new systems are shifting the battlefield, Russia uses something that is borderline classified as a chemical weapon and NATO makes strong statements about no fly zones if there's further provocation etc etc.
Over 3-6 months I can see the tit-for-tat escalation getting to the point where NATO forces are massing in Poland and tactical nukes are being mentioned seriously. But 1 month just seems too fast to get to that point.
Or to put it another way, I would predict that the month before a nuclear strike actually takes place, forecasters will have put much much higher probability on it happening the next month than these numbers.
The weirdest part of the forecast to me is that "Kiev falls by April" declined from 69% to 14%, while "3/6 cities fall by June" effectively didn't change from 71% to 70%.
If you put the two of them together, it seems as those the market's prediction is "Russia wins slowly" instead of "Ukraine holds". Or maybe these markets are behaving inconsistently with each other?
That's one of the issues I have with prediction markets - the very specific phrasing can make a huge difference on outcome. As you note, April isn't the same as June, so they may be asking different questions about timing. It also could be Kiev specific, as in they think Kiev will hold even while three major cities fall.
What I think about these results is that they were both a proxy for "Russia wins" until it became untenable to believe the specific claim (Kiev by April) and the question switched from proxy to the real question. I think this speaks to a different but related problem in prediction markets, that humans do a bad job of parsing details in questions that don't matter much to them. I wonder how many people in those prediction markets even know about the city of Kiev (major city, capital of Ukraine, rough location) when making their predictions? Similarly, how many knew anything about the other major cities?
I'm quite literate when it comes to geography, and although I knew about Kiev, I knew maybe 1-2 other cities (Odessa, and vaguely remembered Mariupol from 2014 news coverage). I'm pretty sure I'm well into the top 10% of Ukraine geography knowledge within the US. How that stacks up with prediction market users, I have no idea. The idea that people were making fully educated predictions based on the location of cities and the Russian armies seems unlikely to me.
I would hope that people at least pull up on map and look at where the cities are before betting money or even internet reputation of a question.
I'm still surprised that "Russia wins" is just as likely now as it was at the beginning of the war. I don't think that it should have fallen by as much as the first question (and the first question is probably currently too low), but I think that it should have fallen some.
I think some people looked at maps and saw that Kyiv is relatively close to the Russian border and because of that expected it to be taken quite quickly.
But I was in Kyiv a week before the war started. I traveled to Chernihiv, a city close to the Russian border. Papers were telling back then that Kyiv will be taken in 48 or 72 hours. I am no expert in the military, but driving around there, looking at the roads, topography, it just seemed very hard to believe that it can be achieved so quickly.
In Kyiv, I also met with one foreign military attaché and asked about the same things. He told me that from the military point of view it is absurd to think that Kyiv can be taken in a few days.
"I suspect, on the balance, the most likely outcome is probably the messy one: Russia loses in that Putin will brutally overpay for whatever fig-leaf concessions he gets, but Ukraine doesn't win in that they either lose territory, access to Europe, or some degree of both."
I’m going to echo what a lot of people have said and say there’s nothing weird about the monotonic decline in the “Fall of Kiev” market. The key new information that has emerged each day is “Russia has failed to make meaningful progress towards capturing Kiev”.
Early on, you could ascribe that to temporary setbacks that would get worked out. But as time goes on and the advance remains stalled, it becomes steadily more likely that no amount of regrouping/resupplying is going to be enough.
Those bombs are generally called atomic, not nuclear which is where we are now. The latter are based on a fusion reaction and are in the mega ton range, rather than the kiloton range of fat boy. Russia has about 1,500 nukes active right now, and a large second strike capability. Most are aimed at the US.
There aren’t enough nukes to destroy the whole world these days, the USSR used to have 40,000 nukes. So none will be fired as South America, China etc. The US will lose most of its major cities and never recover. China will probably come out as the post war hegemon.
I don't think so. I find Russia's second-strike capability to be highly dubious. That's primarily what SSBNs give you, and the Russians have few SSBNs, fewer at sea, fewer still with any real chance of not being sunk within 10 minutes of the opening of hostilities by an American attack submarine presently shadowing it. If you're referring to land-based mobile launchers, this I'm also skeptical about, since in these days of superb orbital reconnaissance I would guess their locations are measured to the nearest meter every morning and fed into targeting computers just in case.
Sure, the science is much more contested than some other climate models. The short story is, the models that predicted nuclear winter also predicted some consequences when Saddam torched oil fields during the Gulf War, but none materialized. It is also a bit speculative how much and how extant the firestorms nuclear airbursts would cause.
The history of blog has seen more detailed arguments about the topic here in the comment section, but I can't find link to relevant comments.
1. There is a military conflict involving countries representing in totality at least 30% of world GDP or 50% of world population in any year in which the conflict is ongoing.
2. At least 10 million people (civillians or military personnel) have been killed in the conflict.
30% of GDP is far more likely than 50% of population. The US alone is 25% of GDP (https://www.visualcapitalist.com/visualizing-the-94-trillion-world-economy-in-one-chart/) so US vs China would qualify, but then again so would US plus a few large allies vs anybody. Even the Iraq War reached the 30% threshold since US + UK + Australia + Poland + Iraq is just enough to exceed that threshold. (Interestingly, US vs Russia wouldn't meet the threshold on its own, but it's hard to imagine US vs Russia without some European allies being involved.)
Ten million deaths is of course a much higher threshold. But the good news is that a war that kills ten million people is barely killing one thousandth of the population, and the distribution of death tolls is weighted towards the low end. So even if there's a 20% chance of such a war before 2050, you're more likely to die of something else in the meantime.
One possible explanation for the forecasts updating monotonically - I (top 60 Metaculus forecaster by standard rankings, top 30 by points per question) use deference (to what extent depends on the question) to the community forecast as an input to my forecasts, so my inside view forecast of Kiev falling was lower, but my actual all things considered forecast updated down as the community went down.
Your highlighting this here gives me something to consider in future, thanks!
I asked about Augur on a previous Mantic Monday, in particular the question of "what happened to Augur;" from the response and other research alongside, it seems like Augur was doing a lot of fancy engineering around zero-knowledge proofs to make their markets work as a "trustless" system, and that this lead to long development times, high transaction fees, a token that was subject to standard crypto pricing weirdness, and so on.
If this is a fair assessment, it makes sense to ask if trustless-ness is a required property for a crypto prediction market. Would it make sense, instead, to use (regular, old, analog, boring, passé) trust? For instance, have a market system where users can register themselves as "judges," post something to the blockchain to the effect of "Hi, I'm [moderately known person], I promise to make obvious assessments of market resolution in exchange for small fees," and then market-openers can assign judges they deem trustworthy to resolve their questions (in exchange for said fees.) One of the key virtues of blockchains is that everything is by-default open, so it should be easy to see when an arbiter does something dumb or malign, whereupon you simply stop trusting them (and they stop earning judge fees.)
Obviously there's a few (!) details that would have to be worked out in such a system, but would scale to at least Metaculus-like scales, and not require any super-arcane proof engineering. Is there a giant flaw in this approach that I'm missing? Is anyone working on anything like this?
Exactly right, my post was indeed directly inspired by Friedman's book. : )
(I was actually thinking of the Icelandic system, but I guess the Somali one works along the same lines. I don't know if the Somali system has any thrilling sagas dramatizing their legal disputes, however.)
Author of the article Scott cited here (https://taosumer.substack.com/p/on-decentralized-prediction-markets?s=w). For clarification, Augur fixed the high transaction fees issues with Augur Turbo which was set up on the Polygon Network, like Polymarket. Augur's token, REP v2, is only for people who want to resolve markets (which represent a small minority of the userbase). Its price does not affect bettors, which are not impacted with crypto price weirdness since you could place your bets in stablecoins... like on Polymarket.
Your approach reminds me of liquid democracy, which definitely is a concept that could (should?) overlap with prediction markets. You're basically substituting financial incentive (the risk of losing your tokens if you lie) to social incentive (the risk of losing your social reputation if you lie). I have to admit that if I was one of the "judge", there is a $$$ threshold above which I'll be willing to sacrifice my Internet reputation for.
I am extremely excited about getting an updated Biodeterminist's guide to pregnancy, I used he current un-updated version over the last year and it was a great starting point for deciding which interventions to pursue. But... while you are at it, I would be even more excited about a biodeterminist's guide to parenting (since that is, as of this instant, even more relevant to me)!
If every day Kyiv has an independent 1% chance of falling, the pattern of "will Kyiv have fallen by April 30th" is pretty much a straight line over time. Good luck arbitraging that. Another way to look at it is "given that Kyiv is still standing, how likely is it to fall in N days", and each day you subtract one to N and use the exact same model. It will constantly yield this monotonous curve. It does not hint at anything suspicious to me.
So far as I know an "efficient" market is merely one that prices accurately in all available information. But obviously if the available information changes, as it does when actual events actually happen, then the efficient market changes with it.
Id est, if actual events steadily reduced the probability of the Ukrainian invasion succeeding -- and there are all kinds of obvious mechanisms by which that could, and probably has, happened, e.g. a steady stream of Russian casualties, the regular loss of general officers, the declining logistical resources of the invaders -- then the efficient market will indeed have a visibly steady trend, but this trend will not obviously have been predictable in the beginning.
Indeed, there are scads of phenomenon which trend steadily until they don't, such as stock market rallies or slumps, or case loads during an epidemic, or fads and bubbles, meaning the underlying dynamics are chaotic, and any impression given by a short-term steady trend that the dynamics are smooth and predictable is illusory.
I think you also conflate "efficient" with "omniscient". A market that prices in all available information may still fail to predict the future, because the information necessary to do so doesn't exist yet. The only way you could have a market where that is *never* true is if you believe in a determinism so strong that a mere handful of variables could predict everything perfectly -- no free will, obviously, but also no large random events, and no chaotic dynamics. Even fairly simple natural systems don't behave this way, so it seems exceedingly unlikely something as complex as a planet of 7 billion conscious thinking creatures could ever be anywhere in the distant neighborhood of that deterministic.
> if actual events steadily reduced the probability of the Ukrainian invasion succeeding -- and there are all kinds of obvious mechanisms by which that could, and probably has, happened, e.g. a steady stream of Russian casualties, the regular loss of general officers, the declining logistical resources of the invaders -- then the efficient market will indeed have a visibly steady trend, but this trend will not obviously have been predictable in the beginning.
This doesn't really matter, because the downward trend in the market is perfectly predictable anyway. It may not be true that actual events steadily reduce the probability of the invasion succeeding. But it is very obviously true that actual events steadily reduce the probability of the invasion succeeding by April 1st. Every day, April 1st gets one day closer.
Yeah that too. But even if you pick a day that is always 20 days out, I suggest the trend line for "Does Kiev fall 20 days from today?" would be steadily downward.
So far, yes, but it wouldn't take much to get it to move upwards again. Once I hear that there's major fighting on the outskirts of the city then I'd move my prediction upwards significantly.
I wouldn't necessarily. I read an interesting essay by a Hungarian in which he suggested the Ukrainian tactics at this point might be sort of "Napoleon vs. Captain Winter" in the sense that they are generally ignoring armor and organized troops (save for the occasional symbolic pyre to show on CNN) in favor of wrecking command and supply infrastructure. The idea being to allow the Russians to strand themselves inside a very large country with perhaps plenty of tanks and troops, but no fuel or food, and inexperienced or absent battalion-level leadership -- Napoleon at the gates of Moscow, more or less -- which seems a good recipe for despair.
Pure speculation, of course, but it does sort of resonate with the noteworthy skill of the Ukrainians at the psychological aspects of this war so far. I'm kind of surprised they had it in them. Normally Russians are good at this kind of patient sadistic mindfuck, it's weird to see the tables turned.
Anyway, if you always buy on the good news and sell on the bad, you're pretty much guaranteed by the Goddess of Fortune to never make money.
> But it is very obviously true that actual events steadily reduce the probability of the invasion succeeding by April 1st. Every day, April 1st gets one day closer.
I can’t tell from the phrasing if you mean this to apply to the specific invasion IRL, or in general. I agree with you if we’re talking about what we actually see in the actual invasion.
In the general case, of course, April 1st gets one day closer every day, but the invading army could get one or more days closer to the target each day, and/or do other things that raise the chance of success, which can change the shape of the probability trend to constant or raising. Or they can be pushed back, which would lower the trend even more.
It's true in general. It doesn't matter that there are events that can raise the probability. The passage of time always lowers the probability of the event occuring by a fixed deadline. The effect of the passage of time is not the same concept as the total effect of everything that might happen.
The contribution of time passing does lower the probability (at least, as long as the event has not happened yet), I agree.
But your comment I replied to says “it is [...] true that _actual events_ steadily reduce the probability” (emphasis mine). That is _not_ true, an “actual event” can raise the probability, as you said.
I wish Metaculus had a stop-loss system that works like this: If the consensus moves more than X% from where it was when I made my prediction, I automatically withdraw my prediction and get an email notification and decide if I want to re-predict using whatever new information became available.
Right now there is unfortunately no way to opt-out of the obligation to continuously update predictions. Maybe at some point in the past I had some extra insight about a topic and wanted to make a prediction using it, but now circumstances have changed and I don't think I'm especially well informed and I want to defer to the community without logging in every day to mimic the community prediction. I just want the ability to withdraw my prediction.
If neough people used stop losses, would it not result in a relatively small number of people potentially causing very big changes? If a group of people changed their mind a lot, it would trigger the predictions of stop-losses to be withdrawed, which would then greatly amplify the initial change.
People could configure X so it doesn't annoy them with cancellation emails about every minor change. Markets don't usually ocillate up and down 20% repeatedly. I think this feature would make people more willing to participate in markets where they don't want to commit to constantly updating, so it could increase the number of *recent* predictions.
Why? The point of the site is not to give points, but reward accurate predictions. Stop-loss would incentivize giving overconfident predictions, because if you make a bad bet it doesn't count.
BTW you do get ability to buy email reminders whenever community consensus significantly changes with tachyons fairly early on.
Currently points are based on a time-weighted-average of the point values of all of your predictions on the same market. This would not change. You would still be penalized in the same way for overconfident predictions. What would change is that you could change your prediction to "no prediction" when circumstances change, and then get zero points for the time that you had no prediction. But you would still be on the hook for the point value of the time spent having a prediction. Opting for "no prediction" is approximately the same as repeatedly updating your prediction to the community consensus, which is what I do on some markets that I have stopped caring about, except "no prediction" would avoid information cascades.
I recognize your problem now, and your "no prediction" suggestion makes more sense. As long the time spent in the market would count, making a prediction results in putting some reputational skin in the game you can't opt out from. But I suspect combining it to automatic stop-loss orders would allow gaming the scoring system in unanticipated ways.
Here is a completely different idea, but might also alleviate your issue. Currently Metaculus point scoring system favors users who are keen to follow the latest news and update their predictions accordingly (on a great deal many of questions). This doesn't necessarily reflect well on the predictors ability to predict the future. (Questions like "does X happen before April 1, 2022" become quite a bit easier close to April 1 we get.) Retroactive closing of questions helps a little, but does not fully fix it (if retroactive period is too lenient).
What if Metaculus (or any other high quality platform who wishes to copy the idea) had a relative points system, where as far as point scoring rule is concerned, your predictions are compared to predictions made about the same time, ie within a set sliding time window? In other words, your ranking would be relative to all other predictors who predicted on same questions as you did about the same time as you did.
The steady decline is consistent with perfectly updated priors, imperfectly updated priors, or constant, deterministic probabilities that need no updating.
It could be perfectly updated because every day probably doesn't see something like "the Russians didn't take over Kyiv today, they must have a terrible army and will lose", but rather, "the Russians were unable to take over the airport today, they will have to instead take over the radio station which is a little easier to defend and doesn't quite provide the level of supply of the airport, so let's adjust our probabilities down a little bit". If they fail to take objective #2, then the probabilities turn to Kyiv falling if/when the Russians take objective #3. The next day's predictions are changed only slightly because the update isn't going from total win to not win, but rather, from winning the best versus winning the next best (conditional on losing the best), and so on.
But note that even a non-changing probability curve could look like that curve. If the probability of taking over Kyiv is coin flip uncertainty, with a changing coin every day regardless of conditionality from the previous day, say, 10% on day 1, 9.5% on day 2, 9% on day 3, etc. etc, then one would expect forecasters to produce that kind of steadily declining graph as "high probability coins" get taken out of the forward looking product.
Would a subsidized prediction market work to extract secrets from Russian officials? You would create a market with the following properties:
1. All the questions ask about stuff that only a Russian official would know for sure until the very end
2. The questions are narrow enough that a Russian official with a sufficient level of clearance would win a ton of money, but also wide enough that others will be encouraged to make bets
3. The odds/payouts will be pre-determined in advance and the market movements won't be revealed until the end, to eliminate the problem of Russians seeing the market and choosing to do something else
4. The payouts will be subsidized by the Western governments so that people could make unlimited bets if they want to
Examples of questions:
1. (asked on Jan 1st 2022) On which exact day will Russia invade Ukraine? (1 to 50 payout if you get the exact day right)
2. On which week will Putin be removed from power? (1 to 100 payout if you get the exact week right)
3. Which city will the Russian army attack next? (1 to 20 payout)
4. On which week will Putin announce he's signing a peace treaty with Ukraine? (1 to 20 payout)
5. What will be the name of the Russian President on January 1st 2024, conditional on this not being Putin? (1 to 100 payout, predefined values + free form available)
If you don't place a limit on how big the market can get, would this be enough to entice well connected Russians to spill the beans? Thanks to the general public making bets too this might not even cost too much money for the West to operate.
A problem with prediction markets for long term bets is that they freeze up money that could be earning some larger return elsewhere. If you're buying a 50% chance of $1M in a year, that's worth far less than $500k, since a 100% chance of $1M in a year is worth less than $1M right now. If you want to have a prediction market that after a year some unlikely event will/won't have happened - you would want that bet to be as good as alternative uses of that money, otherwise no one will take the nearly certain side of that bet. However, that seems trivial to solve by combining the prediction bet with some other financial instruments.
In essence, the simple solution is that the bet should not be denominated in dollars, but in some reasonable investments - e.g. US treasury bonds, or a major stock index ETF - IMHO that should make long-term bets much more attractive. Is this something that has been offered by some prediction market, and if not, why?
This has been brought up a few times, and you've given the generally-agreed-upon solution. Right now prediction markets aren't big enough to try something like this. They're either not for real money, or they're too small and scrappy to handle something like this. The only exception is Kalshi, which is very new and very heavily-regulated.
Suppose England is playing France in soccer, with ten minutes left on the clock and England up 1-0. There's some kind of (idealised, high-liquidity) prediction market going on for "Will England win?"
If the participants in the prediction market can only see the scoreboard, then the market should move slowly and monotonically in one direction as the clock runs out, unless someone scores a goal. But if they can see the field, then the market will move up and down in response to every kick, moving down slightly every time France gets the ball and down slightly every time England gets it, suddenly shooting up every time there's a shot on goal and back down again when it's intercepted.
Monotonic movement is an artefact of the lack of reliable information coming out of the war zone. If metaculus predictors had access to hourly satellite photos then I'm sure the market would fluctuate a lot more.
"This is almost monotonically decreasing. Every day it’s lower than the day before. How suspicious should we be of this?" -- not suspicious at all? Every day the time left until April is shorter than the day before, so of course all other things being equal that probability should be roughly proportional to the time left until April. (Note that the full question text says they mean by April *1*, not by April 30.)
Also, I think there are good ways of using prediction markets and bad ways of using prediction markets.
And I think that "Hey, these guys did pretty well on a prediction market that one time, let's declare them to be Superforecasters and listen to everything they say from now on" is not a good way to use prediction markets.
My thoughts on v1 of the biodeterminist's guide to pregnancy:
1a. The benefit of fish is partially because it's a proxy for iodine status. Foods that come from saltwater are high in iodine and foods that don't come from saltwater are low in iodine [98% confidence] (edit: with some exceptions in modern times for chicken, cattle, and some crops that are artificially provided with iodine)
1b. Some other important minerals may also be easier to find in seafood than in landfood. [90% confidence]
1c. Throughout history coastal peoples tended to outperform landlocked peoples and iodine is probably a significant part of that. [80% confidence]
2a. Unlike saltwater fish, fish oil does not contain adequate amounts of iodine [99% confidence]
2b. The underperformance of fish oil pills is partly due to rancidity. Some of the nutrient decays to a defective form that is probably toxic. [80% confidence]
3. Losing 2-3 IQ points per decade of paternal age was surprisingly high. The rate of de novo mutations is only about 2 per year of paternal age. This implies a cost of about 0.125 IQ points per mutation. I would not have expected a number that high, considering there are 6.4 billion letters in the genome and almost all de novo mutations are SNPs. It suggests you can turn an Einstein (IQ 150) into an imbecile (IQ 50) by changing 800 bases at random, or about 1.25e-7 of the genome. Compare: 1% ethanol increases mutation rate by 10^-8 per base per generation in S Cerevisiae. ( https://www.nature.com/articles/s41467-020-17447-3 ) Let's make the crazy assumption that the effect on yeast is the same as the effect on humans and calculate how many mutations we'd get from being at the legal limit BAC for the entire human growth cycle (30 generations of cells). 6.4*10^9*30*0.08*10^-8 = 153. That's only 20 IQ points. But we're probably less resistant to alcohol than the yeast that made the alcohol. Would like to find some alcohol induced mutation rate data for humans even if it's just HeLa cells in a test tube.
5. Studies on alleged benefits of alcohol are confounded not only by the sick-abstainer effect, but also probably by the has-enough-disposable-income-to-waste-on-alcohol effect, the has-enough-friends-to-go-out-drinking-with effect, and the alcohol-industry-pays-for-the-research effect.
6. Imagine tobacco companies in 1970 found some small real benefit of smoking and hyped the crap out of that. Then it all gets confounded by doctors recommending light smoking to reduce the risk of obesity, and then people who follow doctors' advice start doing that, and the tiny benefit starts looking bigger. They could have actually done this by emphasizing the fact that smoking protects against obesity. I am suspicious that something very similar is going on with alcohol.
Correction: in the present day, iodine is artificially added to the feedstock of chickens and cattle. But historically, you couldn't get very much iodine from any land-based foods.
The nuclear-war probability forecast seems unfortunately low. Looking at the source, it looks like they tried to estimate the probability of a nuclear war as a result of this crisis, took that as a "current probability per year", and then took the one-month probability as 1/12th of that. The current crisis will almost certainly be either over, or stalemated, in much less than a year.
If that understanding is correct, then the truncated-aggregate of 0.067% per month really means "0.8% chance of Russia-NATO nuclear war as a result of this crisis", which, OK, is more plausible but most of that is going to be concentrated in 1-2 months.
Roughly, take the Metaculus prediction of Russia taking <3 major cities by June as the probability of Russia not being able to win a conventional war against Ukraine at a price Putin can afford. There's a possibility of Russia winning in a prolonged war despite their crappy logistics, but there's also the probability that they take 3 cities by June because they nuked or gassed them in April, so handwave those as roughly cancelling.
That's a 30% chance of Russia not winning a conventional war against Ukraine, which would be catastrophic for Putin. Possibly existentially catastrophic; note the 20% forecast of him not running Russia in a year. So, if he can't win a conventional war, 50% for him to try an WMD war (against Ukraine, not NATO). But only 20% for "WMD" meaning "nuclear". So, 0.3 x 0.5 x 0.2 = 3% for Putin to use tactical nuclear weapons against Ukraine sometime in this crisis.
That's going to put huge pressure on NATO to do something drastic, and put Russia on hair-trigger alert for the possibility of NATO doing something drastic at the same time, and that gets us into Cuban Missile Crisis territory. I think we can manage that with maybe an 80% probability of avoiding an actual Russia-NATO nuclear exchange, but that's still a 0.6% probability of "World War Three" coming out of this - close enough to EA's 0.8%.
The 82% for "we'll at least stop short of mass city-killing" and 75% for "if it does come to city-killing, we'll see it in time to run" seem about right.
Good news is, I think the 78% fatalities in London (or Berkeley or any other major city) is on the high side, probably by a factor of two or so. This isn't the Cold War, with tens of thousands of strategic warheads ready to go. I don't think he can spare more than 6-10 warheads in the 100-500 kT range for London, with everything else he'll have to deal with; NukeMap seems to be broken right now but think ~40% fatalities rather than ~80%. Picking the low-hanging fruit in the densely populated urban areas rather than trying to flatten every suburb while simultaneously digging out every deep reinforced-concrete basement.
So, BOTE, six times the near-term monthly risk of World War Three and half the expected casualties if we do get that, or 72 micromorts.
Doubled if you do live in a densely-populated urban area, and quadrupled on top of that if you aren't prepared to evacuate on either very short notice or very modest levels of escalation. So do keep your NorCal friend's number on speed-dial.
Not taking three major cities does not necessarily imply that war continues until June while Russia is unable to take them. That would be imho deep under 30 %, given that from those cities, Russia had already taken one (Kherson), and second seems in imminent danger of falling (Mariupol).
Another situation when this question resolves negatively is if there is an armistice before those cities fall.
Why would Russia nuking NATO or NATO nuking Russia not involve nuking cities? I'm not seeing them using nukes short of general war and in a general war strategic bombing is pretty useful.
The fatalities swing around pretty wildly depending on what counts as dying to nuclear war and what you think is going to happen following a nuclear war; there'd plausibly be a lot of deaths from "didn't Duck and Cover, got burns/glass cuts, because everyone else also failed at this and the hospitals got EMPed/somewhat damaged there's no medical capacity so died of tetanus/gangrene" and "the water supply broke, died of dehydration". That said, if we're conditioning this on being well-informed and at least mildly prepared, those mostly go away.
Nuking cities per se is generally a dumb strategy, militarily speaking, and quite possibly politically as well. Cities are soft targets, but also huge, and by themselves pose only a diffuse and non-immediate military threat. So normally your most important targets are enemy weapons installations -- including of course ICBM silos, but also command centers, radars, submarines that might be in their pens, and airbases that might have strategic bombers (or air-superiority fighters) on them. You might spare some for the seat of government, if you have leftovers.
To the extent the military targets of importance are located *in* big cities, the cities get hit, too, but in perhaps a surprising number of cases, they aren't. In the US all the big ICBM fields are a long way from any cities of consequences. Offut is near Omaha, but Omaha isn't very big. Submarines are based in Kiings Bay, which is somewhat near Jacksonville, and Bangor, which is quite close to Seattle. Barksdale is kind of out in the middle of Louisiana.
> Along with reassuring me I made the right choice not to run and hide...
Sorry, I'm confused: how come ? The table above this quote shows that there's a 0.8% chance/year of global thermonuclear war -- an even which will likely wipe out our technological civilization, if not for good, then for a long time. This probability is way higher than most other X-risk estimates, including those from AI and asteroid strikes. I am kind of tempted to run and hide, if not immediately, then at least soon -- why aren't you ?
Well, the same table predicts that if there _is_ a nuclear war then there's an 82% chance it won't destroy London, and it's hard to see the end of technological civilisation if London is still standing.
And London is one of the most likely cities to be hit, so I assume that the 82% contains a significant chance of a nuclear exchange that doesn't escalate to destroying cities.
Even if London and Moscow and New York are destroyed, nobody has enough spare nukes to start targeting random cities in non-nuclear countries far from the action. Nobody is going to be nuking Santiago, or Kuala Lumpur, or Auckland.
Since the "fallout kills everybody" and "nuclear winter kills everybody" scenarios no longer seem to be taken seriously, I think technological civilisation would survive any nuclear war.
"Run and hide" is an appropriate response for something that hurts people in cities, but is much more manageable for people in rural areas. To the extent that the worry is "wipe out our technological civilization", that seems less likely to make "run and hide" a relevant response.
Presumably the thought is that the first week of unexpected Ukrainian resistance is the most likely specific moment for an internal coup. If he has lasted this long, then there's still risk for him, but it's probably lower and only slowly growing.
Is there a way to sell on Scott's manifold biodeterminist thing? I bought a few of the options but it would be easier if I could directly sell one or two that are (in my opinion) overpriced.
Also, Scott, do you want me to predict what I think you will think as opposed to what I think is *true*? Clearly the market incentivizes me to do the former, but I don't mind doing the latter and losing a little bit of fake money for the sake of contributing to the knowledge pool.
That is to say, if I think intervention X is very important but I also think that you don't think so, then it would be irrational for me to buy X on manifold if I'm trying to maximize returns, but it would still be rational to buy X if I want to encourage you (or others) look into X properly. Is this type of reasoning encouraged or discouraged?
20 % chance on Polymarket that Zelensky will stop being president by April 22 seems too high. I'd short that to something like 10 % if I could could be bothered to figure out how Polymarket works.
Ad WW3 on Metaculus, it was 14 % before February 24 compared to 21 % now. I don't have a good framework on what is the correct probability of WW3, but 7 point jump because of the invasion of Ukraine seems clearly excessive.
EDIT: Also more than 50 000 dead civilians in one city(!) on 12 % seems way to high.
I was surprised to see some commenters questioning the value of markets that give probabilities. We've had these in sports for centuries. The parimutuel odds tell you a lot about a horse race, and make it much more interesting. In football they use a point spread instead so the odds are always 50% but the event is tilted. Again, that's very useful information for a spectator. There are a few bookmakers out there who have to digest the odds in order to make a living, but for most people knowing the odds is part of being an informed spectator.
A reasonable summary of the forecasters vs. experts seems to be 'They're about the same but the experts have making coherent up-to-date predictions as a low priority so they half-ass it while the forecasters take making predictions seriously' Which makes a lot of sense if the forecasters are listening to the experts and translating what they're saying into coherent predictions.
The prediction market I really want to see is where the Ukranian war will be at the end of the year.
I think the point about the "natural 20s" is backwards. My model is, on any day, Russia wins if they roll a natural 20. Each day that they fail gives them one less chance to do so by April 1. Ukraine keeps rolling a natural 1-19, which is exactly what we should think is most likely on any particular day, but it does give us a little bit more information.
> 5: Related: I’m testing Manifold as a knowledge-generation device.
I raised the question in an Open Thread, but are there any successful examples of them being used as a "future-generation" device? Given big enough volumes, prediction markets are open to "insider trading", where you can make use of hidden information (e.g. something that didn't happen yet, but what you're premeditating).
I'm glad to see the edit in the first section. This is a really common pattern in a prediction market with an end date and default outcome. Each day that the thing doesn't happen, the probability will shift closer to 100% chance of the default outcome barring a major event (which will often lead to it dropping to 0%). Things can change as events occur but if you wake me up on March 31st and say "Kyiv hasn't fallen. What are chances it holds until April 1st?" obviously I'm going to assign a higher probability than I would on March 16th, which is higher in turn than March 1st. But I still won't say 100% without new information.
On second thought, you know why this post is so weird? Because Risk! If Russia (well, Ural) invades Ukraine and drops to a 40% chance of winning (say, ~20 attackers to ~25 defenders), the likeliest outcome is that the probability monotonically drops towards 0 (well, not quite monotonically, but with fluctuations very much like this one).
There are 8 instances of "risk" in the posts and another 9 in the comments and Risk literally put Ukraine on the map, and nobody pointed this out?
Jumped straight down the comments when I read "London is a hub for the effective altruism community". Is it?! Can someone tell me more? I just moved to London, and have always been quite sad that due to certain factors tying me to the UK I wouldn't be able to get involved as much as I'd like. How do I find out about/get involved in EA stuff in London?
One possible rational explanation for probabilities falling so consistently is that the distribution of expected evidence is very skewed. To see why, consider a prediction market for "will aliens appear by 2050?". What would the price of this event look like? Realistically, it would start off at some number X, and then slowly decrease in a linear fashion: in 2030 it would be X * 2/3, in 2040 X * 1/3 and finally converge to zero by 2050. The only possible alternative is if in some year, aliens actually do come (or strong evidence of aliens appears), and the number shoots up really high.
The Russian invasion isn't _exactly_ like that, but there are many aspects of it that lean in that direction. The military situation now is roughly "stable", and every day that it remains stable is good for Ukraine. But there's always the possibility of some kind of significant breakthrough for Russia, and that could happen suddenly (if they launch a major offensive, and that offensive succeeds). Of course, there is also the greater possibility of minor new info favorable to Russia coming in, as well as Ukrainian breakthroughs (like the recent breakthrough at Mykolaiv: https://www.dailykos.com/stories/2022/3/17/2086493/-Ukraine-update-Ukraine-retakes-75-miles-of-Russian-held-territory-pushes-toward-Kherson ), which would make a "rational" probability trajectory look more Gaussian (and so if the probabilities update linearly despite these facts, that's evidence of irrational updating-too-slowly).
It may well be a combination of both of these effects.
https://smoothiex12.blogspot.com/2022/03/well-well-well.html how about including some very interesting endnotes from William Engdahl's latest missive (he wrote Full Spectrum Dominance) and I consider him a foremost historian and truthteller all within one's own life prism and experience of course; check out the above blog on Ukraine and Russia with regard the biological weapons. FYI and IMO only. Also, I agree with some of the people on this "site" like in the absence of any meaningful information why not just GUESS? sort of Blinken.. just make it up and pray no one notices. :) have a great Easter week, maybe I will be back next week: I don't like hidden or international sites with codes in it: who knows what could REALLY BE GOING ON?.. :) cheers !
At least our opinion polls have better calibrated probability estimates!
But you are correct, though the issue is a bit subtler. For example, Metaculus advertises its track record: in past Metaculus aggregated predictions have been quite good, so it is worthwhile to look at their current predictions, too. However, they don't guarantee that the predictor pool stays stable nor even of equivalently high quality. Their platform is vulnerable to Eternal September.
The other part of answer is: do not underestimate power of the opinion polls. They are very useful, and they are not cheap to conduct either!
The true public opinion is valuable information which is difficult to obtain and access to it was restricted only to those who have the means to obtain it. And many people have an incentive to manipulate people's perception of what it is. How would you know what is the true aggregated common public opinion if you had only old-fashioned media available? You could talk to people you know, but that is only your social bubble, it could be biased. Suppose you watch TV and read newspapers. A reporter goes out and interviews the proverbial "man on the street" -- they are likely to interview several, and make a decision which interview gets airtime or is printed. Or talk show host invites several pundits to discuss what is the expert consensus on some phenomena or another -- they have editorial discretion over who they invite. And so on. You could call Gallup and ask for their rates -- not something everyone can do.
Regarding the degradation of Zelenskyy NO on Polymarket, a lot of these markets behave like options: they have a time value, and in absence of any new meaningful information, the time value falls until it reaches 0.
Good point! Given this time-value-of-options model, does it surprise you that the Zelenskyy NO price fell from 63% to 20% in the first 14 out of the 53 days until expiration?
It does not. I was a decently sized player in helping the odds go from ~60% to 20 or so. The bulk of the move was when market participants realized how inefficient Russian troop movements have been in Northern Ukraine.
Furthermore, the bulk of the shock move was done by one participant (a whale) who's trades move polymarkets substantially.
I see. Sounds like "market participants realized how inefficient Russian troop movements have been" was a bigger factor than "options generally go to 0 due to time value" in this case -- does that seem right to you in explaining what happened in the market?
Appropriately, "Will Kyiv fall by April" has dropped by more than half since March 7 (one week ago), whereas "Will Kyiv fall by June" has fallen only a little since then.
Excellent description
Does anyone remember Intrade? Very similar dynamics occurred regularly in many of its markets.
This 100%.
For a more colloquial explanation think of a question which goes "Will [Biden] [be president] by [December 31st]?" Particularly this works for questions when the answer is yes for today and you're asking if it'll change to no.
Every day that passes the odds should go down right? So looking back we could say (as Scott does), "this is decreasing linearly, better frontrun it". However this would be a mistake. More likely than small independent events which 'boost' Biden still being president (for Kyiv this would be small events which are pro-Ukraine) it is the lack of a large event which boosts Russia.
We can also think about a lottery. "Will I win the lottery before x date buying a ticket every day?" This would go down over time and is a much easier to understand and simpler model.
It's dangerous to immediately price in the updates you expect to see based on the trend! The forecasters are behaving as proper Bayesian agents and updating slowly, according to the trickle of evidence. Moreover, waiting makes existing evidence stronger - nobody debunked it, you've seen nothing that contradicts it despite time in which such information could surface.
"This is almost monotonically decreasing. Every day it’s lower than the day before. How suspicious should we be of this? If there were a stock that decreased every day for twenty days, we’d be surprised that investors were constantly overestimating it. At some point on day 10, someone should think “looks like this keeps declining, maybe I should short it”, and that would halt its decline. In efficient markets, there should never be predictable patterns!"
If you plotted a prediction for "will this iodine-131 nucleus have decayed by April 1" you'd also get a roughly linear decline (unless it decayed in which case it would jump up to 100%). Prediction markets are allowed to have "story arcs", so long as the *expected* change is zero.
My father's example was the Peso market. For a long time, investing in Pesos gave you a higher return than investing in dollars, which looks like a failure of efficient markets. Then Mexico devalued the Peso.
Similarly here. The pattern is what you would expect if, every day, there was a high probability of a small piece of evidence in favor of Ukraine and a very low probability of a very large piece of evidence in favor of Russia.
I am not a physicist, but IIRC the readioactive decay is exponentially distributed, i.e. "memory-less". It is of course true then, that you would expect this pattern for the iodine-131, but for a different reason than in the case of the war. The process of iodine-131 decay is Markovian because of its memorylessness (is that a word?) but the process of "who is winning the war in Ukraine" is not (the current state of the war is not enough to tell you less than knowing also the history).
But I was convinced by David's example with the peso and his translation to the war in Ukraine. I think it is a good description of how the market probably behaves and it kind of simplifies the situation to a Markov process where you have two absorbing states (Russia winning vs Ukraine winning in the sense of keeping its independence and Zelensky surviving). The process started somewhere in the middle (closer to the Russian absorbing state...pun intended) and has been moving to the other state since.
With a very high probability the process moves incrementally in the direction of the Ukrainian state and with a low probability it can move abrubtly in the Russian direction (from each state it can move directly to the Russian absorbing state or at least close to it, whereas it can only move incrementally in the opposite direction). The closer it gets to the Ukrainian state, the lower the probability of moving to the Russian state (but it can still jump there quickly if the trainsition does occur).
I guess this model makes sense. It seems that Ukraine is ready to fight for quite a long time provided that Russia does not do something very dramatic to force capitulation (capturing Kiev or something...I am not even sure that killing Zelensky would stop the Ukrainian determination at this point). Slow consistent Russian gains seem less likely, they have loads of problems with logistics, low morale, mounting problems at home (I don't think they would be willing or effectively able to fight until June, incidentally). So with that in mind, the markov model above makes sense.
I just wonder whether my description actually adds anything atop of what David said or whether it is not simply the same thing described in a more complicated way :)
What makes you say the progress of a war is necessarily non-Markovian? I would say good arguments can be given for it being a pretty Markovian process. That doesn't at all mean it's memoryless in the sense that anything at all can happen now, it just means its trajectory is determined mostly by what happened recently (with perhaps some constant bias in one direction or another). That seems rather likely to describe the course of a war, in the sense that what Belligerent A or B does at the start has a big influence on what happens next, and what happens next (and the next set of decisions) have a big influence on the next step, and so forth -- classic Markovian model.
A strongly non-Markovian model would be something a like a self-avoiding random walk, a situation where certain decisions made very early in the conflict have profound and somewhat random influence on things that happen much later. Like a bomb planted at time X that goes off at some unpredictable time X + dX (dX >> 0) and which then has a decisive effect. It's difficult to see how this could readily manifest in human affairs. What would this be? The chance of foreign intervention? Death of some critical leader? It would seem to presuppose a pretty contingent view of the evolution of a war, that "surprising" things out of the distant past could have unpredictably decisive effects on the present -- "unpredictable" in the sense that we could not easily trace their influence through time, A at time t1 caused B at time t2 which caused C at time t3, et cetera, *because* such a chain of influence is exactly what a Markovian process represents, a *non-Markovian* process is one where there are weird "Star Trek transporter" connections by which something far in the past leaps over all intervening events and affects the present all by itself, without going through any causal chain.
I think the discussion of Markovian vs non-Markovian is a bit of a non sequitur. I believe that the relevant property here is being a martingale not a Markov process. Obviously for continuous time (continuous) martingales there is not going to be any period of monotonicity almost surely, but in the case of discrete time, it is easy to see that for any delta > 0 there are martingales such that with probability at least 1 - delta the process is monotonically increasing and remains bounded between 0 and 1.
The martingale property comes immediately from mathematical finance, where a fundamental theorem says that no-arbitrage implies the existence of a measure under which the price is a martingale. Even ignoring the finance part, though, it should be obvious that we expect a (Levy) martingale. If we want the ERM predictor of the indicator of an event (with squared loss) with some information available in a sub-sigma algebra, we just take the conditional expectation. Thus at each time t, we should predict the expectation of the final event conditioned on the information available up to time t. This is just a Levy martingale. Where does the Markovian property enter?
Hmm, I guess you are right. The conditional probability should be a martingale. I guess this is the crucial bit that explains the behaviour.
I was thinking that a harmonic function of a Markov process is a martingale and the probability of ending in an absorption state of the markov process is a harmonic function.
I guess that Carl is right though that the Markov property is not that unreasonable here. What we really have at time t is not just the current state of the war but also all the previous information as well. And the bet is made based on this full information so it should trivially be Markovian.
I do not think it needs to be Markov; there could certainly be path-dependent information and my earlier comment still applies. Even absent the relevance question (I still do not think that it matters one way or the other for this question on the behavior of the process) I do not think that it is obvious that it even should be Markov, even in an efficient market.
I agree that it does not need to be Markov. My comment is rather justifying the model in case it is Markov, but you are correct that it applies more broadly.
You are saying that the reason is A whereas I was saying that it is because of B (and B implies A). You are correct that A is the important thing and there are ways you can arrive at A other than B.
Still, B still seems like quite a reasonable model to me after reflection (after thinking about Carl's comment).
But maybe I am missing a more elegant model which could describe the war that you might have had in mind?
Not sure what your objection is. A martingale *is* a Markov process.
Being a martingale is not synonymous with being Markov either in discrete or continuous time. There are plenty of examples one way or the other.
To give two trivial examples in discrete time, consider a process where X_0 has support of size at least 3 and y_n are independent Rademacher random variables for each n. Let X_n = X_{n-1} + y_n X_0. This is clearly a martingale. It is not Markov though because conditioning on the path (knowing X_0) concentrates the distribution for X_n given the history on two points while not knowing X_0 cannot have this concentration. (It is not hard to extend this example to continuous time by solving an SDE).
For the other way, take any Markov process (X_t) whose distributions conditional on the past are nonatomic and any function f that is subharmonic but not harmonic on the support of the X_t. Then the process given by f(X_t) is going to be a submartingale but not a martingale. This applies both in discrete and continuous time.
[Edited to add example]
Ah I see what you mean. Thanks for the example. It's pretty contrived, however. I think for the real-world processes we are discussing here the distinction is not relevant.
Markov processes and martingales have a close relationship (via harmonic functions, also solutions of "well-behaved" SDEs are Markov processes) but they are indeed not synonymous and neither necessarily implies the other as Adam mentions.
On the other hand a lot of simple examples of martingales are markovian and vice versa (or their functions are, such as the relevant probability above) and a "markov process of the war" I am describing above is still the most intuitive toy model for me.
I agree with you. and I am scientist I need some other statistical algorithms with ten pages of proofs or something IMHO.
Exactly! Basically the pattern speaks to how little we know at a "gears level" about how the war is going. Given the right prior distribution I'd expect the prediction to evolve the same way if our only source of new information was a webcam in downtown Kyiv.
It's worse than the iodine-131 example; it's more like "will this iodine nucleus have decayed by April 1" when you don't even know what isotope of iodine you're dealing with.
I'm kind of OK with Metaculus's initial and current numbers, but the linear-ish trend seems off. The first week gave us a *lot* of useful information. First, it told us that the Ukrainian army wasn't going to fold like the Afghan National Army, that the people would back them up, and that Volodymyr Zelenskyy had the right stuff. All of those were in real doubt on 24 February. Second, it told us that this is not your father's Guards Motor Rifle Division we're dealing with, that the Russian Army of 2022 is not the one we were worried was going to overrun the Fulda Gap in 1982 but has severe logistical, organizational, technical, and morale problems. We *sort* of knew that on 24 Feb, but not with high confidence.
So most of the fall-off should I think have occurred in the first week, ruling out most of the "Shock and Awe prevails" scenarios. Rather like, if I hand you one of the seven major isotopes of iodine picked at random, there's a good chance that it's one of the ones with a half-life of hours to a few days, definitely not going to last a month. But a week later, you pretty much know that you're dealing with one of the longer-lived isotopes and maybe the actually-stable one.
Now I want to spreadsheet up a forecast of the decay of a random iodine atom in one month, conditional on it having lasted N days already.
Unless I'm mistaken, you can easily compute this by hand: Let $T$ be April 1, $t$ be the time of the first occurrence of decay, $p_s=\mathbb P(t\leq T\mid t>s)$ be the probability of decay by time $T$, conditional on no decay up to time $s<T$, $A_i$ be the event that we're looking at an iodine nucleus with half-life $a_i$ for $i=1,\dots,n$.
Then we have $p_s = \sum_{i=1}^n\mathbb P(A_i\mid t>s)\mathbb P(\text{Exp}(a_i)\leq T-s) = \frac{\sum_{i=1}^n e^{-{s\over a_i}}(1-e^{-{T-s\over a_i}})}{\sum_{i=1}^n e^{-{s\over a_i}}}$.
Plugging in some made-up values, e.g. only two different types of iodine atoms, one with half-life shorter and one longer than the time horizon of interest ("shock and awe prevails" vs "Ukraine strong", respectively), we actually get [something linear-ish with a bit of a kink](https://www.wolframalpha.com/input?i=plot+%28e%5E%28-s%2Fa%29%281-e%5E%28-%28T-s%29%2Fa%29%29+%2B+e%5E%28-s%2Fb%29%281-e%5E%28-%28T-s%29%2Fb%29%29%29%2F%28e%5E%28-s%2Fa%29%2Be%5E%28-s%2Fb%29%29+for+T%3D10%2Ca%3D1%2Cb%3D20+from+s%3D0+to+s%3D10).
Intuitively, due to the exponentially decaying tails of exponential distributions, once we observe no decay after 2h we can almost certainly rule out half-times of <1h (provided there is an alternative hypothesis, like the possibility of another particle with half-time >2h). Once the contribution from a particle with longer half-time becomes dominant, linear decay seems reasonable.
For the record, I was primarily interested in how much this particular toy model would deviate from a linear function and would not use this for actual predictions.
(Apologies for the lack of formatting, I couldn't find a way to properly display links/equations on here.)
Shouldn't the fall of Kiev by April decrease with each day closer to the April deadline that it hasn't fallen? It doesn't seem that strange to me
Does it seem strange to you that it fell from 69% to 14% in the first 14 out of 53 days before the deadline?
April 1 is 17 days away, and the forecast graph only goes back to feb 28, 14 days ago, so 31 days total. I don't know where your 53 comes from.
to answer the question, no i don't think that drop is strange. over that two weeks, we saw the results and new information adjusted quickly, then it should slowly decline as the deadline approaches.
Sorry, I'm probably just reading the graph badly, but what period in 2/28 are you thinking of as the "adjusted quickly" period, and what period is the "slowly decline" period? (The graph that Scott posted of the metaculus aggregation over time seems to move pretty smoothly from 2/28 to now; I don't see an obvious "adjusted quickly" period.)
The way I interpret and see the chart, it had some relatively old value Feb 28-Mar5, then steep drop to new lower value, then slow decay from there. This may not be the case, but the main point of my comment is to describe the behavior after that Mar5 drop. The comment from House Always Wins illustrates that point perhaps more articulately.
Ah, thinking of 3/5-3/6 as being a quick adjustment, and the resulting slide from 40% to 14% in the 8 days of the remaining 26 seems helpful. I still wouldn't have guessed that 2/3 of the probability mass "should" lie in the first 1/3 of days, so it does seems like there's more information being incorporated than just "probability decreases with each day closer to the April deadline that Kiev hasn't fallen".
Oh, I mixed up the deadline on the Zelenskyy polymarket market with the Kiev metaculus market. Should have said:
"Does it seem strange to you that it fell from 69% to 14% in the first 14 out of 32 days before the deadline?"
The slowness of Russia in the early phases of seizing Kyiv suggests a slowness of their attack in the later phases. The distribution of "how long it takes Russia to get its troops to Kyiv," "how long does it take Russia to start shelling Kyiv with artillery," "how long does it take Ukrainian resistance to break down," "how long does it takes to seize the first raion," "how long does it take to seize the second raion," etc, are not going to be uncorrelated, even if they don't all have the same distribution.
A prediction market for next week’s Powerball numbers makes no sense as that is an unknowable thing. Similarly how Putin, Zelinskyy, Biden, Sholtz, random software problems that accidentally send a Russian Cruise Missile to Warsaw all interact to influence the future is just as unknowable as the future state of the balls bouncing around at lottery HG.
What am I missing?
A prediction market for next week's Powerball numbers should have every combination priced at around ~0.000000342229781%. The balls bouncing around have a huge number of unknowable interactions, so it's hopeless to form predictions based on predicting every step of the process. But, like with the Powerball *results*, it's possible to estimate what the distribution of *outcomes* might be, even if it's hopeless to predict every footfall of every soldier.
How is the prediction market doing better than our general knowledge of statistics?
If our general knowledge of statistics is the best we can do, then a prediction market should represent that.
It isn't *just as* unknowable as Powerball, even if it's quite unknowable. All those men you named have stated their intentions and have a history of behaviors in situations of varying degrees of similarity. Human behavior can be predictable to some extent. Degrees of confidence exist.
That’s true if you’re predicting the outcome of general events. The life insurance market works great at predicting how likely you are to die in the next 10 or 20 years. It’s predicting specific events that I’m not sure about.
For example - Tesla wants to take out $100 billion in life insurance to protect itself from Elon commuting suicide or other mwise ending his life prematurely. How much value is the life expectancy of the set of all 50 year old males in the US?
It provides a good base rate, which you can adjust upwards or downwards based on Elon Musk's particular characteristics (e.g. is he depressed, does he drink and drive, does he do dangerous drugs, etc).
Superforecasting, by Philip E. Tetlock, covers the specifics of how those who make these kind of "one-off" predictions accurately manage to do so. Unfortunately, it's sufficiently long and detailed in its coverage that there are few money quotes, though somebody reproduced the "Ten Commandments" from the appendix here: https://www.lesswrong.com/posts/dvYeSKDRd68GcrWoe/ten-commandments-for-aspiring-superforecasters
Do we have data that says prediction markets of Musk like scenarios are more accurate than general life insurance underwriting?
Define "Musk-like scenarios"? If you mean scenarios which rely on complex, multivariable human decision making similar to the ongoing war in Ukraine, you can just go to https://www.metaculus.com/questions/track-record/ and look at Geopolitics; Metaculus gets significantly above monkey levels, though I don't know how it compares to braindead heuristics like "the next five years will be identical to the previous five years".
If you mean literally predicting when specific people die, that's not a very common thing to see on prediction markets, probably because of the moral hazard.
A prediction market for next week's Powerball numbers is perfectly legitimate - the correct probability to assign to any number would be 1 / 1 million (or whatever the actual Powerball odds are). The only reason people wouldn't use such a market is that it would be so obvious that nobody would bet against the 1 / 1 million number.
What's the correct odds (ie the equivalent of saying "1 / 1 million") for Russia taking Kyiv this month? I'm not sure, but it's probably a different number than the correct odds for Russia taking Mariupol this month, and it's fair to speculate on both sets of odds.
The fact that we can't get certainty is fine, all prediction markets are asking you for is a probability. You probably have opinions on this probability already - 99% would seem unreasonably high, 0.1% unreasonably low.
Does that answer your question?
The odds of winning any lottery, for anybody, is 50%. It’s 50-50. You either do or you don’t.
That explains why half of all lottery players win.
Wait.
People definitionally bet on Powerball numbers at worse than their actual odds of winning. So in principle there should be people available to bet that the odds of a given combo are higher than they actually are based on gamblers' fallacies or magical thinking.
How would a powerball prediction market work in practice? I’m taking legitimate to mean it would work to determine the outcome.
No one is saying that you could use a prediction market to predict the exact winning numbers. They are saying that the prediction market will give an accurate PROBABILITY for a given set of numbers.
That is, if you ask a prediction market "will all 20 of these dice come up as 6s?" the market will say "with probability 1/6^20", which is the best possible answer given the available information.
To a hypothetical person who doesn't know how to calculate that probability for themself, that would be useful information.
That hypothetical person would have to be significantly dumber than average, since the market merely presents the conventional wisdom. So the question might be: why would a prediction market be useful to someone *other than* the unusually clueless?
I mean, the stock market tells me what the conventional wisdom is on the future earnings of Tesla, and that would be useful if I had never heard of the company. Gee, maybe buying Tesla stock makes more sense than putting my money under the mattress.
But if that does not describe me -- if I know as much as the average schmo -- what useful information can I extract from the present price of TSLA? Pretty much zip. If I buy when the price is high, and sell when the price is low -- i.e. I follow the conventional wisdom the market summarizes -- I will never make money. The *only* way the information can be practically useful is if I have *more* information than the average schmo, e.g. I am insider (or just smarter than everybody else) and I happen to know the conventional wisdom greatly under- or overvalues Tesla. In that case I can use the current market value to know when to bet against the market and win.
I guess I'll add that it is nearly definitionally not possible for my extra insight to dominate the market, so that the market reflects my extra knowledge. The market isn't exactly a one-man-one-vote democracy, but it's also not super duper different from that. Except in the rare circumstances that I am staggeringly rich *as well as* unusually well informed, my influence on the market (even as I make a killing) is not going to be enough to shift the conventional wisdom.
Obviously, the information from the market won't help you **at beating the market**.
If you know something the market doesn't, you can make money, and in the process make the market more informed. If the market knows something you don't, then you can learn something, and use that information ELSEWHERE in your life.
You might even be able to do both of those things on a single question, if you and the market know *different* information. ("I don't know how the market decided on a price of X, but I have inside information I'm sure the market hasn't taken into account, so I can guess the *direction* of its error.")
But for any given bit of info, you can either learn it from the market OR use it to make money from the market, not both. Markets are not perpetual motion machines.
(And even if you were going to try to use the market's information to play the market, "buy high, sell low" would be a profound misunderstanding of what the market is telling you. If you had an option to buy or sell at a fixed price of $X (regardless of what price the market is currently trading at), then looking at the market price could tell you whether you should exercise that fixed-price option. But if you're talking about buying or selling **at the market price**, then a high price is NOT a recommendation to buy--it indicates the value is high, but THE COST IS ALSO HIGH, so they cancel out!)
Sure, but those things are only true pretty far out on the statistical tail. I have to know *a lot* more than the market, because I need to know enough to know I'm better informed than the market. Given the epistemic bias of the rational actor against thinking he knows something special -- he's dead right and millions of others are dead wrong -- it would take a very well informed person to rationally conclude he can safely bet against the market. Similarily, it takes an unusually poorly-informed person to realize the market knows much more than he does, since the market doesn't have any special qualifications -- it's made up of average people. How badly informed do I have to be on any subject to conclude reliably that the averae schmo on the street knows much more than me? Pretty badly.
So in both cases *as a prediction tool* the market is only useful for a small demographic at either extreme. That wouldn't justify its existence, since it would be ipso facto useless for the great bulk of its participants. Why would anyone willingly participate in a market the main positive effect of which is to make unusual people who are not you better off? Nobody's that altruistic.
People *do* participate in "prediction markets," e.g. every time they go to the track and place a bet on the horses. The existence of the odds -- or indeed the market in general -- benefits only the people who know nothing at all about horses, who can avoid putting their money on the 100-1 long-shot, and the people who know *everything* about horses, who can *put* their money on the 100-1 long shot that they happen to know is going to win for some obscure reason or other. The rest of the people participate not because they seriously expect to come out ahead, financially (unless they're idiots), but because people generally get a thrill out of betting on stuff for some weird psychological reason. (Perhaps because by strategic forgetting of our losses we can feel smart about our wins.)
Yes, I agree, if I have a fixed option the market can tell me whether to exercise it. Under what consideration could I have something like that in a prediction market? I am not following how this would work in, say, a prediction market on the Ukrainian war. How could I come into possession of such an option, in a natural way? If you have a specific example in mind, it would help me understand what you mean.
Couldn't this market effective simulate a decentralized lottery? Instead of buying an actual $2 powerball ticket with a 1/1 million chance to win a million dollars, someone buys a $2 share on a specific number. Enough people enjoy the tiny chance of winning a huge amount of money that they're willing to do this for real lottery tickets even with negative expected value. Then all it needs is a bunch of rational actors with enough money to provide liquidity in exchange for profiting from the irrational choices of the gamblers, who would probably end up running automated programs that arbitrage any discrepancy between the current market and the mathematical odds. The total amount of money in the system would vary, but in expectation the return for winning should be approximately the reciprocal of the odds for winning, minus any losses from irrational behavior, which are profited by the arbiters.
This seems like it would end up remarkably similar to a regular lottery, except instead of the lottery provider making tons of money from the expected value discrepancy, multiple potential liquidity providers would compete, driving the profit margins down and creating a lottery that was more fair towards the gamblers.
Depends on whether the purpose is to have fun gambling, or actually predict stuff in a reliable way. If the latter, then the necessary underlying assumption is that the future is already determined -- the die is already cast, and it's just a question of discerning which side will face up before it actually happens, like predicting the path of a thrown rock in a vacuum from its initial position and velocity.
We can fudge it a little bit by saying it has to be predictable only in a statistical sense, like ordinary statistical mechanics, or quantum mechanics, e.g. whether *this* time we get nuclear war is unknowable, but we can say with certainty that the odds are 2.633%. That doesn't really change anything, because the predictability still requires perfectly deterministic underlying dynamics -- no chaos, no random events.
Is it possible that 'story arcs' in prediction markets come because most people have far heavier priors on other people's assessment and judgement, than they do on events in reality itself?
Maybe most people, even those using prediction markets, have a real low prior on something like 'prediction markets are totally wrong here' ? I have far more experience with 'prediction markets being general correct' than i do with forecasting military outcomes.
For 5., the Biodeterminist's Guide to Parenting, #5 reads "Supplement with nicotinamide mononucleotide (for fathers)"
Should we assume that #s 2, 3, 7, 4, 10, all of them really, apply to the mother prior to birth? Subsequent to birth? The baby?
Yes, all the others are about the mother when pregnant.
Do we assume that the mother is already taking prenatal vitamins before these recommendations are given? My guess is yes or else folate/iodine/etc will top the list. This matters for things like vitamin D (already in the prenatal pill, so I guess you mean extra D on top of that) and the vitamin A variants (vitamin A is in the prenatal pill, and too much of it is actively harmful, which I assume is why you list these specific lutein/zeaxanthin variants instead of a generic A pill).
"One possibility is that, by a crazy coincidence, every day some new independent event happened that thwarted Russia and made Ukraine’s chances look better. Twenty dice rolls in a row came up natural 20s for Ukraine. Seems unlikely."
That's more or less exactly what's happened. It's not unlikely at all, it's how we experience the world.
On day 1, Ukraine was still standing.
On day 2, Ukraine was still standing.
On day 3, Ukraine was still standing.
etc etc etc
These are all new information! On any given day N, different things might happen, we don't know. We have to wait and observe. Simply drawing a linear extrapolation through the first few days would be lunacy.
Also, labeling these as "natural 20s" is a huge overstatement of what we know about the underlying distribution. We have very little information on which to base an estimate of "Can Ukraine defend against an attack by the Russian army?", and I offer as evidence the fact that this post exists, we're having this conversation, etc etc. So it *may* be that Ukraine is on a hot streak and rolling 20s, or it may be that the outcomes we see are right in the middle of the distribution because it happens that Ukraine is a better match for Russia than some (many? most? idk!) people thought.
I agree, perhaps the market is approximating a “no info environment”. Suppose I arrange to watch a movie at 3pm with a friend. My friend tells me he will try to get there on the early side, but when I arrive he isn’t there. Now how does the probability of him arriving in time for the movie update every minute? I know literally nothing about his progress save the passage of time. I think it looks quite a bit like the Kiev market.
I agree. See also the thread on House Always Wins' comment above
I was thinking about this problem recently while writing code for estimating the speed of a spinning wheel from a pulse train. It's tricky even if you have an estimate from previous pulses.
Consider a bus stop, where expect the bus to arrive every 10 minutes. Up until 10 minutes, you can estimate time remaining based on your original estimate. But now the bus is late, so what do you do?
One way to do it is to assume the bus is always about to arrive. At 11 minutes your new estimate is that buses arrive every 11 minutes. And so on. When the bus finally arrives, your prediction will be correct.
This seems to result in a smoother graph when the speed of the spinning wheel is slowing down. Adding elapsed time to the estimated period results in the frequency decaying like 1/x.
Sometimes, though, the bus will arrive early, and then you need to revise your estimate up.
This is sort of like saying the Lindy effect applies Ukrainian resistance.
But what bounds can we put on this? I would expect that cities under siege (completely surrounded) can't hold out forever. On the other hand, people say the Russian Army has logistical limits on how long they can continue the invasion.
Who has time on their side? The theory of protracted war is apparently that the Ukrainians win by surviving. Perhaps the markets believe this? Or maybe that's just the structure of this bet, since it has a deadline.
Kiev is not completely surrounded, so that changes your calculus a bit. The Russians have not been able to cut off access to the city from the south, and haven't made much progress in that direction over the past few days at least.
I'm thinking that one of the worst things about surviving a nuclear war would be finding yourself in a society organized and dominated by the kind of people who optimize their lives around surviving a nuclear war.
The vast majority of survivors would be people who survived by some combination of coincidence and general competence, no matter how (if) it happens the vast majority of survivors will not be nuclear preppers.
"vast majority" does not equal "organized and dominated by"
we already live in a disease-surviving, war-surviving hellhole (:
"Preppers are horrible people even if they're right" is enough of a trope in post-apocalyptic fiction that I've become suspicious of it. It reads like status gaming by the big winners of current society (which produces very different values and constraints that they've in turn optimized for) against their outgroup. Selfish, suspicious anti-authority types with unpopular beliefs are exactly the people I'd want to side with in a post-apocalyptic environment.
I've had to interact with too many people like that in real life, you see.
But if they're selfish, why would *they* want to side with you?
Preppers are people who are irrational, but happened to be right by pure coincidence. The fact that they are right doesn't change the likelihood of them being irrational.
If we were invaded by aliens who could be stopped by wearing literal tin foil hats, I'd expect the survivors to be even worse.
This past month has really brought back the '80s, when "is nuclear armageddon more or less likely this week?" was a perfectly normal topic for news discussion.
Not to mention all the Jimmy Carter comparisons
> If there were a stock that decreased every day for twenty days, we’d be surprised that investors were constantly overestimating it. At some point on day 10, someone should think “looks like this keeps declining, maybe I should short it”, and that would halt its decline. In efficient markets, there should never be predictable patterns!
Most high-growth tech stocks have been in secular decline since November (see eg NFLX, COIN, DASH, PTON, U, etc), so if you really believe this, you should go pick up those obvious $100 bills just laying on the ground :)
Ha ha yes that's why I always buy stock when the price is increasing and sell when the price is decreasing, which makes perfect sense except gosh darn it I seem to have less and less money every year...
The efficient market hypothesis appreciates your sacrifice!
No no. When price is decreasing, you don't sell (you are already too late to cash in). When price is increasing, you don't sell (you are too early to cash in).
Question from the audience? Let me repeat that everyone hears it. Gentleman here in the studio asked, "When do you buy"? The answer is, whenever you have sufficient funds, or at random intervals, sir.
This is the strategy that allowed us, BUY'N'HODL Inc, to finance our fancy interstellar home, the Axiom. We will return to planet Earth after it has been sufficiently cleaned.
Another explanation for the slow decline in Ukraine predictions:
This might happen if people are thinking something along the lines of "Russia will not capture Kiev by April unless they do something to turn things around before then". If so, then every day that Russia fails to turn things around provides a bit more evidence that Russia will not capture Kiev. This doesn't violate conservation of expected evidence or the efficient market hypothesis, because every day there is some small chance that Russia WILL turn things around and the odds will swing substantially in the other direction.
Yep, this seems to be the consensus answer to the mystery (several other threads discuss it too).
Nice handle, by the way.
I'm kind of echoing what's been said before in the comments, but I'm really not sure why the monotonically decreasing pattern is odd. A stock shouldn't decline over time, but a stock is also in theory an infinite prediction (there's no time expiry). A prediction with a specific deadline is quite different.
Suppose your mental model is something like "there is a 1% chance Kiev will fall on any given day". With 30 days until April, you'd predict about a 74% chance of Kiev not falling (.99 ^ 30). With 15 days left, conditional on it not falling yet, you're up to about an 86% chance. Your base rate probability of Kiev falling doesn't have to change (it's still 1% on any given day), but the decreasing time to expiry changes your odds of the actual contract resolving one way or the other.
I basically agree with this take. Every day Russia fails to make progress in taking Kyiv is a day closer to the deadline. They might still be making progress, but so far it's too slow for them to make an April 1st deadline. I think their best bet is in surrounding the city and starving out the defenders, but that'll take a couple of weeks even after they've surrounded the city, which they're still far from doing.
I think that the other thing is that people have been expecting the Russian army to figure out its organizational problems but they haven't done that yet. The longer they fail to do that, the more it looks like they won't be able to do it. Which means the war will be continue to be slow-going and costly for them.
I think the key here is that the evidence has mostly been of the following form:
"the Russians haven't moved very far, and they've already been invading for X days"
That's evidence that becomes monotonically stronger as X increases but there's no sharp cutoff point where it qualitatively changes in character. It's quite different from the updates you would get from crucial pivotal events (agreeing to ship a more advanced SAM system, the fall of a city, losing a key skirmish etc) but I also think it's extremely relevant evidence.
For instance, if we think one of the two situations is likely to be a correct description of the situation, we could reasonably believe that every day without solid Russian progress is evidence towards option 2 and update accordingly.
Scenario 1) Russian forces are fundamentally powerful enough to take Ukraine but take time to deploy effectively
Scenario 2) Russian forces are not powerful enough to take Ukraine
Every day X ticks one higher, and the P(evidence| scenario 2) becomes higher and P(evidence|scenario 1) becomes lower meaning a larger update but incremental update towards scenario 2.
Even now, it's still possible that the Russians are gradually getting into position and will still prevail, but the alternative story where their logistics, morale, tactical awareness etc are too constraining to achieve their objectives has become a lot more plausible.
Yes, this was a weird objection to me. The monotonically decreasing with time in the absense of major new developments seemed the obviously correct reaction.
Zvi has very similar behavior in his Covid predictions. For example, before the recent Hong Kong surge, as time went on with no (apparent) changes in China's case load, he gradually incremented up the likelihood that they would successfully contain the disease. Then, something major happened (the Hong Kong spike and various smaller outbreaks) and he dramatically reduced the likelihood.
(This was a comment made by the real Enopoletus Harding on Metaculus.)
I thought of leaving here due to my disagreements with the moderation here, but since Scott has mentioned me on his blog (I can't comment there since I'm banned for extremist posts), I will continue to make predictions here. Needless to say, the chances of Russia taking Kiev go down each day there is no advance. I personally don't think it makes sense to update very rapidly, since, for all I know, the Russians may have been using the past two weeks to prepare some lightning assault -not something that's likely, certainly, but the chances of that are too high to simply exclude off the bat. Maybe I should update more rapidly -I've been underperforming the community when evaluated at all times. But knowing when to update and having good starting positions are not easy skills to learn.
I've also created an account on Manifold Markets, I expect my predictions to be better there (I have a habit of putting the vast majority of my money on a few markets I can easily win in):
https://manifold.markets/EnopoletusHarding
No, I'm not trolling. Wrong, maybe. It's happened before. I had 10% chance of a Russian invasion for way too long, but I obviously changed that as the evidence became more clear.
Remind me what your disagreement with the moderation here was?
(asking because I thought it was that I banned you, but you're here, so I must be misremembering)
Supposedly, "here" is Metaculus, not ACX. https://www.metaculus.com/accounts/profile/118219/
More seriously: "is EHarding a troll" is an excellent example of a question not suitable to a prediction market.
That doesn't quite work. NotEH says the comment was on Metaculus, and says EH was banned on Scott's blog, which is here.
It does not seem implausible to me that Harding would also have disagreements with the Metaculus moderation.
This is correct, he is referring to the Metaculus moderation
Ukraine is going to lose in the end. Maybe only after most cities are destroyed, but they cannot win. Then they will be ruled by russian marionettes & dictat in the foreseeable future. The dictat will be harder and longer, in particular on the West-ukrainians, the more blood the Russians have to spill to conquer the country. The only alternative is for the ukrainians to cut their losses some way, but so far that seems unlikely. I'm writing this because the endgame is what matters. Making "predictions" about which week Kyiv and other cities fall is a teenager-way of thinking and acting about a real-life war. It is of very marginal importance which week it will happen, and such speculations are also rather distasteful. Grow up.
The big questions rely on and derive from the little questions. The most accurate way to make large predictions is to break them down into many smaller subpredictions for which we can derive some reference class. Or, to borrow Jesus's phrasing, "Whoever can be trusted with very little can also be trusted with much."
The longer Kyiv holds out, the more time there is for some major event to change the entire equation. A general collapse in one of the two armies, for example. Of for another major power to get directly involved, for another. The math might currently be in Russia's favor (actually it's not that clear; it depends on what the casualty numbers are really like,) but there's still time for major events to upset matters.
It's also important to understand that it's not just the "endgame" in Ukraine that matters. This war affects most of the world at the moment, and whether Ukraine continues to fight matters a lot to what the other powers involved should do about it. For example, if Ukraine can be expected to hold out for another few months or a year, then Putin needs to worry about supply and domestic issues so it affects him greatly. It also affects the kind of aid it makes sense for the EU and NATO to send to Ukraine. It also affects what countries that used to depend on Ukrainian grain will have to do to deal with their own crisises. So yes, this matters.
"Teenager"? It seems like you can make real money betting against the people you disagree with. There are lots of adults who making a living on speculative markets.
My mental model for the progress of this war is essentially as a race - damage to Ukrainian civilians and military may push Ukraine to concede a disadvantageous peace, and damage to Russian military and economy may push Russia to concede a disadvantageous peace; both of these damages are progressing at varying speeds, and the question is which one of these "progress bars" will reach a breaking point first.
And I strongly disagree with your assumption - I don't see how Russia can force *that* kind of peace even if their military operation completely succeeds now. If Russia conquers or destroys all the major cities and installs a puppet government, that does not enable Russia to rule over Ukraine - the populace will not agree to that, not even in the eastern Russian-speaking areas like Kharkiv, not after what has been done in the war. And Russia can not afford enough manpower to enforce proper control, that would require something like 500k permanent garrison there; so Russia will have to leave and their marionette regime falls right afterwards (as after USSR-Afghanistan). I mean, pacifying and controlling a hostile country is really, really hard; it has not gone well for USA recently despite immense resources, and even for the quite effective Stalin-NKVD apparatus it took many, many years after WW2 to suppress guerilla warfare in Eastern Europe - and those people and Taliban didn't have external support/arms supply from all the wealthy countries, like Ukrainian guerillas would have in case of a Russian conquest. So as Russia can't plausibly hold Ukraine, they would have to concede to something more reasonable (e.g. recognition of Crimea + DNR/LNR, perhaps a bit more land, but Ukraine government remaining in place) even in the case of an overwhelming victory - which itself is far from certain.
In my opinion the main part of the war has not started yet - the Ukrainian strategy apparently was to trade space for time, preparing for the actual fighting in taking over the large cities, which is going to be difficult for the attacker, and where the war would be decided. So far Russia is only assaulting the relatively small Mariupol (Kherson and Melitopol are many times smaller than that), not even trying to make serious progress in Kharkiv or Kyiv. We can see how much men and time it took Russia to take Grozny; it seems plausible to consider that Kyiv - which is much larger and has much more resourced defenders - will take at least as much.
Im skeptical of the "russia will never submit Ukraine". Didn't they do it with Chechnya pretty succesfully? Seems like they could pull it off at least on a portion of Ukraine.
Chechnya is much smaller, but Ukraine doesn't have much in the way of mountains.
Chechnya is a very relevant example of how they were very much *not* able to do it "pretty successfully". They tried to enforce submission for many years and failed, and they succeeded only with a combination of keeping excessive military garrison (at levels which Russia can't afford at the larger scale of Ukraine) plus effectively buying off the local leaders and population with excessive subsidies (again, at levels which Russia can't afford at the larger scale of Ukraine) and political concessions.
The Chechen resistance did not end with surrender and submission, the current result is "an understanding" where formally Kremlin gets a victory, territorial integrity and declarative submission, but in practice devolves severe autonomy and large monetary "tribute" to Chechnya. Kadyrov is not like other regional leaders which are at the mercy of Moscow, he is independently powerful, with his own army (that is not accountable to Russian hierarchy), and is effectively an ally of convenience to Putin personally and has a lot of de-facto autonomy from the rest of Russian authorities - there are all kinds of incidents with Kadyrov's people in violent conflict with other Russia region's court system and police system, and having de facto immunity for that. I don't have a specific English source in mind, but if you're interested about this aspect, I would recommend you to read about how Kadyrov came to power, all of that process was quite interesting.
Thanks, I will further look into this. Any reliable books on the subject?
I don't know any - it's relatively recent, most written information is in Russian with little analysis from the western part, and I doubt if the authorities would allow to publish an analytical book like that in Russia (and IIRC both threats and actual violence has been applied both to local Chechens and expats commenting on Kadyrovs regime). However, I just stumbled upon a quite extensive summary in English in Twitter-thread form - https://twitter.com/kamilkazani/status/1497612331953577991
I'm not sure if your claim is "I'm so obviously right that it's useless for people to try to predict things" or "the things you're predicting are too small scale to matter".
I disagree with the first because the people who say the first have a worse-than-average track record of being right, and I disagree with the second because big things are made up of small things. I bet there are a lot of Ukrainian generals who would like to know in what month Kiev will/won't fall right now, and would change a lot of what they're doing if they knew.
But also - realistically everything that's happening on prediction markets now is capacity-building / proof-of-concept. The more we establish a record for prediction markets on small things, the more likely we'll have them ready and established when we need to use them for big things.
(the Arb report I linked finds that forecasters beat epidemiologists and models in predicting the spread of COVID-19 and its effects on interventions - seems potentially useful)
I would like to go on record that I find that poster's attitude distasteful AND I disagree with his object-level prediction about the end-result of the Ukraine war.
However, upon reflection, he may be on to something, and I don't think your answer addresses that something. A more sophisticated, steelmanned version of the sentiment he expresses would go as follows:
It should be, of course, uncontroversial that every little data point brings us closer to a correct big picture view. However, it does not follow that a *prediction market* about every little single data point gives us more information. Realistically, prediction markets about little things will be extremely correlated with each other, influenced much more strongly by the overarching big picture sentiment looming above them than by any actual new information directly related to the question they're trying to answer. (And even if one of them is actually pricing in some important niche knowledge, we have no way of teasing that knowledge out from the single probability number that the prediction market produces.)
In other words, in a flow of information from primary data to their increasingly more processed derivatives, prediction markets about small things may actually be further down the line than the big picture view, with the big picture view having already absorbed the relevant information from the primary small-thing datapoints. The corollary to this would be that predictions about small things should not, and probably do not, reflexively influence the big picture view. This is an important, non-trivial point that, honestly, I find likely to be correct.
> they cannot win
It is not very likely or likely, but it remains possible (Putin gets overthrown or something).
Making claims like that is a perfect example why tools like prediction markets can be useful.
I'm actually trending towards thinking that Russia is not likely to win unless China props them up. At this point it's looking like a long, long grind and I don't know how long Russia can sustain that with the way their economy is going.
this is an example of definitions being really important - as the definition of 'win' is the key factor here. And where prediction markets on those various definitions of 'win' are highly valuable
Perhaps the best outcome for Russia is a 'loss' in Ukraine.
A Russian 'win' leaves Russia maintaining a very large military force in the country for years. Possibly decades. A never ending money and resource pit. This was true from the moment it become clear that Ukrainians were not hoping to become part of Russia.
Especially in a scenario where Russia trades their withdrawal for significant sanction relief. But even in a scenario without that, I think Russia withdrawing would give some portion of parties participating in sanctions the cover they need to slowly back away.
Why do you think they cannot win? Russia is about three and a half times the size of Ukraine in population, which is an advantage but may not be enough to balance the facts that Ukraine is on the defensive, that the Russian military appears to be very poor and Ukraine's military pretty good. Russia is richer than Ukraine, which in most contexts would be important, but Ukraine is being supplied by NATO countries that are much richer than Russia.
It's hard to be sure but it looks as though, despite its size, Russia has a very limited number of trained soldiers, many, possibly most, of whom have already been committed to the invasion. So I think one plausible outcome is that the Russian military ends up unable to advance further, possibly unable to hold what it is currently occupying. They can kill a substantial number of Ukrainians, but not a substantial fraction of the population, and at the cost of further losses of men and material.
"Ukraine is going to lose in the end. Maybe only after most cities are destroyed, but they cannot win."
Your confidence in the ability of the Russian army to sustain its morale and current operational tempo indefinitely, is I think unjustifiably high. The best unclassified sources I can find suggests that the Russian army has already lost 10-20% of its *total* effective combat strength in this campaign. That can be reconstituted, but not in real time. And Putin isn't going to run his army down to literally zero combat effectiveness.
The 30% of forecasts where Russia hasn't even taken three major cities by June, those are the ones where Ukraine probably wins. And, yeah, maybe promises to recognize Crimea as part of Russia to dissuade Puting from spite-nuking Kyiv, but that's still a win for Ukraine.
"Russia will probably win" is a reasonable forecast. "Ukraine *can't* win", is not.
> One possibility is that, by a crazy coincidence, every day some new independent event happened that thwarted Russia and made Ukraine’s chances look better.
This doesn't require Thursday. The "event" is just the absence of any big Russian breakthrough.
If Ukraine is stronger than anyone could observe, then each day of non-defeat reveals this fact just a little bit to a rational observer
Regarding expert aggregation, the University of Chicago does this for economics, a bit. Link: https://www.igmchicago.org/igm-economic-experts-panel/
I'm not sure I follow the statement "At some point on day 10, someone should think “looks like this keeps declining, maybe I should short it”, and that would halt its decline."
It doesn't make sense to me that prediction markets would improve estimates without additional causal information (information tied to actual "facts on the ground") vs. this notion of the market just predicting based on itself, which seems antithetical to the goals of meaningfully accurate predictions. Beyond the market forces at play (which are just a tool to motivate effortful predictions out of forecasters), "technical analysis" (as opposed to fundamental analysis) adds little to the causal information included in prediction markets' final estimates, correct (unless someone believes that the best forecasters are always leading major shifts/trends rather than lagging them, which seems questionable; why couldn't poor forecasters move the market?)?
This feels like a fundamental limitation of the current approach/emphasis on forecasting that I've never been comfortable with: trying to predict something important by staring at a stock ticker rather than looking at the facts on the ground (even given those facts are fuzzy/noisy and their interpretation is also fuzzy/noisy). Moore's Law held for the past 50 years. Therefore, based on that, many futurists (as far as I can tell) just extrapolate out that growth in hardware performance and cost indefinitely into the future rather than ask whether there are any concrete reasons to believe semiconductors can keep being shoved into dramatically smaller form factors (Moore's Law has failed to keep pace for about a decade as far as I understand). It feels like this idea of "the trend is moving this way, therefore it will continue" is fundamentally antithetical to the whole point of forecasting and serves just to reinforce current market predictions. "a real dog" points out that delayed/lagging Bayesian updating strategies are useful, stabilizing heuristics, which I'd agree with although this a form of arguing that we should be biased to towards our previous priors more than we would otherwise be (if we didn't assume that new info is inherently less trustworthy than older info). That said, this feels different from trying to predict new things (rather than old things that were presumably based on causal info) based on a trend. The Kyiv trend looks kinda linear; in 2 weeks it could go down to -50%!
In this case it seems like the prediction market forecast is too biased toward prior predictions given how rapidly the situation is changing, as Scott pointed out. However, it's a different thing to say "given the most recent news [causal information] I've seen, the current market is off and I should short" from "given the trend is moving downward, I should short", which assumes all changes in the market should be accompanied by positive feedback effects (more formally, in linear control systems, this is equivalent to trying to always have the feedback time constants perfectly tuned, which is hard, particularly in unpredictable situations and I'd imagine would lead to instability; how long is a "long trend" that we should predict based on vs. a "short one" that is just spurious noise? How do you know in advance?). The trend only looks clearly linear in retrospect... at least for me, not being an expert of course, the situation didn't feel so linear at the time.
On the nuclear forecasting I do wonder if one month is too short a time horizon for us to expect increased risk. In my mental model of escalation we would see several weeks or months of increased tensions that gradually drag NATO in and make nuclear exchange conceivable. Perhaps Russia bombs a school next week and the west responds by putting the MIGs on trucks and shipping them to Ukraine a week later. Perhaps a missile aimed at the trucks hits a Polish border station and NATO increases shipments. A week later after some of these new systems are shifting the battlefield, Russia uses something that is borderline classified as a chemical weapon and NATO makes strong statements about no fly zones if there's further provocation etc etc.
Over 3-6 months I can see the tit-for-tat escalation getting to the point where NATO forces are massing in Poland and tactical nukes are being mentioned seriously. But 1 month just seems too fast to get to that point.
Or to put it another way, I would predict that the month before a nuclear strike actually takes place, forecasters will have put much much higher probability on it happening the next month than these numbers.
The weirdest part of the forecast to me is that "Kiev falls by April" declined from 69% to 14%, while "3/6 cities fall by June" effectively didn't change from 71% to 70%.
If you put the two of them together, it seems as those the market's prediction is "Russia wins slowly" instead of "Ukraine holds". Or maybe these markets are behaving inconsistently with each other?
That's one of the issues I have with prediction markets - the very specific phrasing can make a huge difference on outcome. As you note, April isn't the same as June, so they may be asking different questions about timing. It also could be Kiev specific, as in they think Kiev will hold even while three major cities fall.
What I think about these results is that they were both a proxy for "Russia wins" until it became untenable to believe the specific claim (Kiev by April) and the question switched from proxy to the real question. I think this speaks to a different but related problem in prediction markets, that humans do a bad job of parsing details in questions that don't matter much to them. I wonder how many people in those prediction markets even know about the city of Kiev (major city, capital of Ukraine, rough location) when making their predictions? Similarly, how many knew anything about the other major cities?
I'm quite literate when it comes to geography, and although I knew about Kiev, I knew maybe 1-2 other cities (Odessa, and vaguely remembered Mariupol from 2014 news coverage). I'm pretty sure I'm well into the top 10% of Ukraine geography knowledge within the US. How that stacks up with prediction market users, I have no idea. The idea that people were making fully educated predictions based on the location of cities and the Russian armies seems unlikely to me.
I would hope that people at least pull up on map and look at where the cities are before betting money or even internet reputation of a question.
I'm still surprised that "Russia wins" is just as likely now as it was at the beginning of the war. I don't think that it should have fallen by as much as the first question (and the first question is probably currently too low), but I think that it should have fallen some.
I think some people looked at maps and saw that Kyiv is relatively close to the Russian border and because of that expected it to be taken quite quickly.
But I was in Kyiv a week before the war started. I traveled to Chernihiv, a city close to the Russian border. Papers were telling back then that Kyiv will be taken in 48 or 72 hours. I am no expert in the military, but driving around there, looking at the roads, topography, it just seemed very hard to believe that it can be achieved so quickly.
In Kyiv, I also met with one foreign military attaché and asked about the same things. He told me that from the military point of view it is absurd to think that Kyiv can be taken in a few days.
I think it's also the choice of 6 cities. Russia taking Kherson, Kharkov, and Mariupol while still being unable to take Kiev is plausible.
A prediction from Bret Devereaux on Twitter: https://twitter.com/BretDevereaux/status/1503457631347879949
"I suspect, on the balance, the most likely outcome is probably the messy one: Russia loses in that Putin will brutally overpay for whatever fig-leaf concessions he gets, but Ukraine doesn't win in that they either lose territory, access to Europe, or some degree of both."
I’m going to echo what a lot of people have said and say there’s nothing weird about the monotonic decline in the “Fall of Kiev” market. The key new information that has emerged each day is “Russia has failed to make meaningful progress towards capturing Kiev”.
Early on, you could ascribe that to temporary setbacks that would get worked out. But as time goes on and the advance remains stalled, it becomes steadily more likely that no amount of regrouping/resupplying is going to be enough.
I cannot wrap my head around Metaculus putting 20% on WW3. Are you telling me everybody on Earth is playing Russian roulette right now?
EDIT: I was so worried when I wrote this that I didn't even spot the pun.
By 2050. And only a few % of the world's population died in WW2, hardly Russian roulette.
WW2 was not nuclear. I'm also wondering: how does such a high number co-exist with the superforecasters' nuclear risk analysis?
But WWII was nuclear. It is the only nuclear war ever to have occurred. WWI is the world war that wasn't nuclear.
It's not a nuclear war if only one party has the bomb. Or at least that's what I meant.
"WW2 didn't involve the excahnge of thermonuclear ICBMs between mutliple states"
Those bombs are generally called atomic, not nuclear which is where we are now. The latter are based on a fusion reaction and are in the mega ton range, rather than the kiloton range of fat boy. Russia has about 1,500 nukes active right now, and a large second strike capability. Most are aimed at the US.
There aren’t enough nukes to destroy the whole world these days, the USSR used to have 40,000 nukes. So none will be fired as South America, China etc. The US will lose most of its major cities and never recover. China will probably come out as the post war hegemon.
I don't think so. I find Russia's second-strike capability to be highly dubious. That's primarily what SSBNs give you, and the Russians have few SSBNs, fewer at sea, fewer still with any real chance of not being sunk within 10 minutes of the opening of hostilities by an American attack submarine presently shadowing it. If you're referring to land-based mobile launchers, this I'm also skeptical about, since in these days of superb orbital reconnaissance I would guess their locations are measured to the nearest meter every morning and fed into targeting computers just in case.
Not sure if trolling.
Are there any modern projections of how many people would die in a hypothetical world war 3?
More then 1 billion in the immediate aftermath. Then it depends on if there is a nuclear winter, and the subsequent economic recession and starvation.
*if* there's a nuclear winter? it's not guaranteed?
Sure, the science is much more contested than some other climate models. The short story is, the models that predicted nuclear winter also predicted some consequences when Saddam torched oil fields during the Gulf War, but none materialized. It is also a bit speculative how much and how extant the firestorms nuclear airbursts would cause.
The history of blog has seen more detailed arguments about the topic here in the comment section, but I can't find link to relevant comments.
The resolution criteria on this one are:
1. There is a military conflict involving countries representing in totality at least 30% of world GDP or 50% of world population in any year in which the conflict is ongoing.
2. At least 10 million people (civillians or military personnel) have been killed in the conflict.
30% of GDP is far more likely than 50% of population. The US alone is 25% of GDP (https://www.visualcapitalist.com/visualizing-the-94-trillion-world-economy-in-one-chart/) so US vs China would qualify, but then again so would US plus a few large allies vs anybody. Even the Iraq War reached the 30% threshold since US + UK + Australia + Poland + Iraq is just enough to exceed that threshold. (Interestingly, US vs Russia wouldn't meet the threshold on its own, but it's hard to imagine US vs Russia without some European allies being involved.)
Ten million deaths is of course a much higher threshold. But the good news is that a war that kills ten million people is barely killing one thousandth of the population, and the distribution of death tolls is weighted towards the low end. So even if there's a 20% chance of such a war before 2050, you're more likely to die of something else in the meantime.
One possible explanation for the forecasts updating monotonically - I (top 60 Metaculus forecaster by standard rankings, top 30 by points per question) use deference (to what extent depends on the question) to the community forecast as an input to my forecasts, so my inside view forecast of Kiev falling was lower, but my actual all things considered forecast updated down as the community went down.
Your highlighting this here gives me something to consider in future, thanks!
I asked about Augur on a previous Mantic Monday, in particular the question of "what happened to Augur;" from the response and other research alongside, it seems like Augur was doing a lot of fancy engineering around zero-knowledge proofs to make their markets work as a "trustless" system, and that this lead to long development times, high transaction fees, a token that was subject to standard crypto pricing weirdness, and so on.
If this is a fair assessment, it makes sense to ask if trustless-ness is a required property for a crypto prediction market. Would it make sense, instead, to use (regular, old, analog, boring, passé) trust? For instance, have a market system where users can register themselves as "judges," post something to the blockchain to the effect of "Hi, I'm [moderately known person], I promise to make obvious assessments of market resolution in exchange for small fees," and then market-openers can assign judges they deem trustworthy to resolve their questions (in exchange for said fees.) One of the key virtues of blockchains is that everything is by-default open, so it should be easy to see when an arbiter does something dumb or malign, whereupon you simply stop trusting them (and they stop earning judge fees.)
Obviously there's a few (!) details that would have to be worked out in such a system, but would scale to at least Metaculus-like scales, and not require any super-arcane proof engineering. Is there a giant flaw in this approach that I'm missing? Is anyone working on anything like this?
Hey look, it's the Somali criminal justice system! ( https://slatestarcodex.com/2017/11/13/book-review-legal-systems-very-different-from-ours/ )
Exactly right, my post was indeed directly inspired by Friedman's book. : )
(I was actually thinking of the Icelandic system, but I guess the Somali one works along the same lines. I don't know if the Somali system has any thrilling sagas dramatizing their legal disputes, however.)
Author of the article Scott cited here (https://taosumer.substack.com/p/on-decentralized-prediction-markets?s=w). For clarification, Augur fixed the high transaction fees issues with Augur Turbo which was set up on the Polygon Network, like Polymarket. Augur's token, REP v2, is only for people who want to resolve markets (which represent a small minority of the userbase). Its price does not affect bettors, which are not impacted with crypto price weirdness since you could place your bets in stablecoins... like on Polymarket.
Your approach reminds me of liquid democracy, which definitely is a concept that could (should?) overlap with prediction markets. You're basically substituting financial incentive (the risk of losing your tokens if you lie) to social incentive (the risk of losing your social reputation if you lie). I have to admit that if I was one of the "judge", there is a $$$ threshold above which I'll be willing to sacrifice my Internet reputation for.
I am extremely excited about getting an updated Biodeterminist's guide to pregnancy, I used he current un-updated version over the last year and it was a great starting point for deciding which interventions to pursue. But... while you are at it, I would be even more excited about a biodeterminist's guide to parenting (since that is, as of this instant, even more relevant to me)!
Maybe that will come later. For now I recommend Emily Oster's books.
Yep! I'm caught up on her work. Thanks for the recommendation though.
If every day Kyiv has an independent 1% chance of falling, the pattern of "will Kyiv have fallen by April 30th" is pretty much a straight line over time. Good luck arbitraging that. Another way to look at it is "given that Kyiv is still standing, how likely is it to fall in N days", and each day you subtract one to N and use the exact same model. It will constantly yield this monotonous curve. It does not hint at anything suspicious to me.
Robin Hanson actually confirmed a bet, though he's just betting $1 to someone else's $100. https://twitter.com/robinhanson/status/1502993000238133250
So far as I know an "efficient" market is merely one that prices accurately in all available information. But obviously if the available information changes, as it does when actual events actually happen, then the efficient market changes with it.
Id est, if actual events steadily reduced the probability of the Ukrainian invasion succeeding -- and there are all kinds of obvious mechanisms by which that could, and probably has, happened, e.g. a steady stream of Russian casualties, the regular loss of general officers, the declining logistical resources of the invaders -- then the efficient market will indeed have a visibly steady trend, but this trend will not obviously have been predictable in the beginning.
Indeed, there are scads of phenomenon which trend steadily until they don't, such as stock market rallies or slumps, or case loads during an epidemic, or fads and bubbles, meaning the underlying dynamics are chaotic, and any impression given by a short-term steady trend that the dynamics are smooth and predictable is illusory.
I think you also conflate "efficient" with "omniscient". A market that prices in all available information may still fail to predict the future, because the information necessary to do so doesn't exist yet. The only way you could have a market where that is *never* true is if you believe in a determinism so strong that a mere handful of variables could predict everything perfectly -- no free will, obviously, but also no large random events, and no chaotic dynamics. Even fairly simple natural systems don't behave this way, so it seems exceedingly unlikely something as complex as a planet of 7 billion conscious thinking creatures could ever be anywhere in the distant neighborhood of that deterministic.
> if actual events steadily reduced the probability of the Ukrainian invasion succeeding -- and there are all kinds of obvious mechanisms by which that could, and probably has, happened, e.g. a steady stream of Russian casualties, the regular loss of general officers, the declining logistical resources of the invaders -- then the efficient market will indeed have a visibly steady trend, but this trend will not obviously have been predictable in the beginning.
This doesn't really matter, because the downward trend in the market is perfectly predictable anyway. It may not be true that actual events steadily reduce the probability of the invasion succeeding. But it is very obviously true that actual events steadily reduce the probability of the invasion succeeding by April 1st. Every day, April 1st gets one day closer.
Yeah that too. But even if you pick a day that is always 20 days out, I suggest the trend line for "Does Kiev fall 20 days from today?" would be steadily downward.
So far, yes, but it wouldn't take much to get it to move upwards again. Once I hear that there's major fighting on the outskirts of the city then I'd move my prediction upwards significantly.
I wouldn't necessarily. I read an interesting essay by a Hungarian in which he suggested the Ukrainian tactics at this point might be sort of "Napoleon vs. Captain Winter" in the sense that they are generally ignoring armor and organized troops (save for the occasional symbolic pyre to show on CNN) in favor of wrecking command and supply infrastructure. The idea being to allow the Russians to strand themselves inside a very large country with perhaps plenty of tanks and troops, but no fuel or food, and inexperienced or absent battalion-level leadership -- Napoleon at the gates of Moscow, more or less -- which seems a good recipe for despair.
Pure speculation, of course, but it does sort of resonate with the noteworthy skill of the Ukrainians at the psychological aspects of this war so far. I'm kind of surprised they had it in them. Normally Russians are good at this kind of patient sadistic mindfuck, it's weird to see the tables turned.
Anyway, if you always buy on the good news and sell on the bad, you're pretty much guaranteed by the Goddess of Fortune to never make money.
> But it is very obviously true that actual events steadily reduce the probability of the invasion succeeding by April 1st. Every day, April 1st gets one day closer.
I can’t tell from the phrasing if you mean this to apply to the specific invasion IRL, or in general. I agree with you if we’re talking about what we actually see in the actual invasion.
In the general case, of course, April 1st gets one day closer every day, but the invading army could get one or more days closer to the target each day, and/or do other things that raise the chance of success, which can change the shape of the probability trend to constant or raising. Or they can be pushed back, which would lower the trend even more.
It's true in general. It doesn't matter that there are events that can raise the probability. The passage of time always lowers the probability of the event occuring by a fixed deadline. The effect of the passage of time is not the same concept as the total effect of everything that might happen.
The contribution of time passing does lower the probability (at least, as long as the event has not happened yet), I agree.
But your comment I replied to says “it is [...] true that _actual events_ steadily reduce the probability” (emphasis mine). That is _not_ true, an “actual event” can raise the probability, as you said.
I wish Metaculus had a stop-loss system that works like this: If the consensus moves more than X% from where it was when I made my prediction, I automatically withdraw my prediction and get an email notification and decide if I want to re-predict using whatever new information became available.
Right now there is unfortunately no way to opt-out of the obligation to continuously update predictions. Maybe at some point in the past I had some extra insight about a topic and wanted to make a prediction using it, but now circumstances have changed and I don't think I'm especially well informed and I want to defer to the community without logging in every day to mimic the community prediction. I just want the ability to withdraw my prediction.
If neough people used stop losses, would it not result in a relatively small number of people potentially causing very big changes? If a group of people changed their mind a lot, it would trigger the predictions of stop-losses to be withdrawed, which would then greatly amplify the initial change.
People could configure X so it doesn't annoy them with cancellation emails about every minor change. Markets don't usually ocillate up and down 20% repeatedly. I think this feature would make people more willing to participate in markets where they don't want to commit to constantly updating, so it could increase the number of *recent* predictions.
Why? The point of the site is not to give points, but reward accurate predictions. Stop-loss would incentivize giving overconfident predictions, because if you make a bad bet it doesn't count.
BTW you do get ability to buy email reminders whenever community consensus significantly changes with tachyons fairly early on.
Currently points are based on a time-weighted-average of the point values of all of your predictions on the same market. This would not change. You would still be penalized in the same way for overconfident predictions. What would change is that you could change your prediction to "no prediction" when circumstances change, and then get zero points for the time that you had no prediction. But you would still be on the hook for the point value of the time spent having a prediction. Opting for "no prediction" is approximately the same as repeatedly updating your prediction to the community consensus, which is what I do on some markets that I have stopped caring about, except "no prediction" would avoid information cascades.
I recognize your problem now, and your "no prediction" suggestion makes more sense. As long the time spent in the market would count, making a prediction results in putting some reputational skin in the game you can't opt out from. But I suspect combining it to automatic stop-loss orders would allow gaming the scoring system in unanticipated ways.
Here is a completely different idea, but might also alleviate your issue. Currently Metaculus point scoring system favors users who are keen to follow the latest news and update their predictions accordingly (on a great deal many of questions). This doesn't necessarily reflect well on the predictors ability to predict the future. (Questions like "does X happen before April 1, 2022" become quite a bit easier close to April 1 we get.) Retroactive closing of questions helps a little, but does not fully fix it (if retroactive period is too lenient).
What if Metaculus (or any other high quality platform who wishes to copy the idea) had a relative points system, where as far as point scoring rule is concerned, your predictions are compared to predictions made about the same time, ie within a set sliding time window? In other words, your ranking would be relative to all other predictors who predicted on same questions as you did about the same time as you did.
The steady decline is consistent with perfectly updated priors, imperfectly updated priors, or constant, deterministic probabilities that need no updating.
It could be perfectly updated because every day probably doesn't see something like "the Russians didn't take over Kyiv today, they must have a terrible army and will lose", but rather, "the Russians were unable to take over the airport today, they will have to instead take over the radio station which is a little easier to defend and doesn't quite provide the level of supply of the airport, so let's adjust our probabilities down a little bit". If they fail to take objective #2, then the probabilities turn to Kyiv falling if/when the Russians take objective #3. The next day's predictions are changed only slightly because the update isn't going from total win to not win, but rather, from winning the best versus winning the next best (conditional on losing the best), and so on.
But note that even a non-changing probability curve could look like that curve. If the probability of taking over Kyiv is coin flip uncertainty, with a changing coin every day regardless of conditionality from the previous day, say, 10% on day 1, 9.5% on day 2, 9% on day 3, etc. etc, then one would expect forecasters to produce that kind of steadily declining graph as "high probability coins" get taken out of the forward looking product.
Would a subsidized prediction market work to extract secrets from Russian officials? You would create a market with the following properties:
1. All the questions ask about stuff that only a Russian official would know for sure until the very end
2. The questions are narrow enough that a Russian official with a sufficient level of clearance would win a ton of money, but also wide enough that others will be encouraged to make bets
3. The odds/payouts will be pre-determined in advance and the market movements won't be revealed until the end, to eliminate the problem of Russians seeing the market and choosing to do something else
4. The payouts will be subsidized by the Western governments so that people could make unlimited bets if they want to
Examples of questions:
1. (asked on Jan 1st 2022) On which exact day will Russia invade Ukraine? (1 to 50 payout if you get the exact day right)
2. On which week will Putin be removed from power? (1 to 100 payout if you get the exact week right)
3. Which city will the Russian army attack next? (1 to 20 payout)
4. On which week will Putin announce he's signing a peace treaty with Ukraine? (1 to 20 payout)
5. What will be the name of the Russian President on January 1st 2024, conditional on this not being Putin? (1 to 100 payout, predefined values + free form available)
If you don't place a limit on how big the market can get, would this be enough to entice well connected Russians to spill the beans? Thanks to the general public making bets too this might not even cost too much money for the West to operate.
That sounds like a cheap way for the Kremlin to feed you disinformation.
How do you stop their president from shooting anyone who participates?
A problem with prediction markets for long term bets is that they freeze up money that could be earning some larger return elsewhere. If you're buying a 50% chance of $1M in a year, that's worth far less than $500k, since a 100% chance of $1M in a year is worth less than $1M right now. If you want to have a prediction market that after a year some unlikely event will/won't have happened - you would want that bet to be as good as alternative uses of that money, otherwise no one will take the nearly certain side of that bet. However, that seems trivial to solve by combining the prediction bet with some other financial instruments.
In essence, the simple solution is that the bet should not be denominated in dollars, but in some reasonable investments - e.g. US treasury bonds, or a major stock index ETF - IMHO that should make long-term bets much more attractive. Is this something that has been offered by some prediction market, and if not, why?
This has been brought up a few times, and you've given the generally-agreed-upon solution. Right now prediction markets aren't big enough to try something like this. They're either not for real money, or they're too small and scrappy to handle something like this. The only exception is Kalshi, which is very new and very heavily-regulated.
You could do this with play money too by denominating the bet in play stockmarket fund.
Lido Finance tries to tackle this issue in DeFi with what they call liquid staking, it's an interesting perspective
Suppose England is playing France in soccer, with ten minutes left on the clock and England up 1-0. There's some kind of (idealised, high-liquidity) prediction market going on for "Will England win?"
If the participants in the prediction market can only see the scoreboard, then the market should move slowly and monotonically in one direction as the clock runs out, unless someone scores a goal. But if they can see the field, then the market will move up and down in response to every kick, moving down slightly every time France gets the ball and down slightly every time England gets it, suddenly shooting up every time there's a shot on goal and back down again when it's intercepted.
Monotonic movement is an artefact of the lack of reliable information coming out of the war zone. If metaculus predictors had access to hourly satellite photos then I'm sure the market would fluctuate a lot more.
Tired: betting on the war outcome
Wired : betting on the fog-of-war opacity
"This is almost monotonically decreasing. Every day it’s lower than the day before. How suspicious should we be of this?" -- not suspicious at all? Every day the time left until April is shorter than the day before, so of course all other things being equal that probability should be roughly proportional to the time left until April. (Note that the full question text says they mean by April *1*, not by April 30.)
Also, I think there are good ways of using prediction markets and bad ways of using prediction markets.
And I think that "Hey, these guys did pretty well on a prediction market that one time, let's declare them to be Superforecasters and listen to everything they say from now on" is not a good way to use prediction markets.
My thoughts on v1 of the biodeterminist's guide to pregnancy:
1a. The benefit of fish is partially because it's a proxy for iodine status. Foods that come from saltwater are high in iodine and foods that don't come from saltwater are low in iodine [98% confidence] (edit: with some exceptions in modern times for chicken, cattle, and some crops that are artificially provided with iodine)
1b. Some other important minerals may also be easier to find in seafood than in landfood. [90% confidence]
1c. Throughout history coastal peoples tended to outperform landlocked peoples and iodine is probably a significant part of that. [80% confidence]
2a. Unlike saltwater fish, fish oil does not contain adequate amounts of iodine [99% confidence]
2b. The underperformance of fish oil pills is partly due to rancidity. Some of the nutrient decays to a defective form that is probably toxic. [80% confidence]
3. Losing 2-3 IQ points per decade of paternal age was surprisingly high. The rate of de novo mutations is only about 2 per year of paternal age. This implies a cost of about 0.125 IQ points per mutation. I would not have expected a number that high, considering there are 6.4 billion letters in the genome and almost all de novo mutations are SNPs. It suggests you can turn an Einstein (IQ 150) into an imbecile (IQ 50) by changing 800 bases at random, or about 1.25e-7 of the genome. Compare: 1% ethanol increases mutation rate by 10^-8 per base per generation in S Cerevisiae. ( https://www.nature.com/articles/s41467-020-17447-3 ) Let's make the crazy assumption that the effect on yeast is the same as the effect on humans and calculate how many mutations we'd get from being at the legal limit BAC for the entire human growth cycle (30 generations of cells). 6.4*10^9*30*0.08*10^-8 = 153. That's only 20 IQ points. But we're probably less resistant to alcohol than the yeast that made the alcohol. Would like to find some alcohol induced mutation rate data for humans even if it's just HeLa cells in a test tube.
4. Wikipedia has a handy chart of the dose-response effect of alcohol on various diseases: https://en.wikipedia.org/wiki/Alcohol_and_health#/media/File:Health_risks_of_alcohol_consumption.svg The tiny bit of CHD hormesis for very light drinkers is dwarfed by the very bad effects on cancer at all levels.
5. Studies on alleged benefits of alcohol are confounded not only by the sick-abstainer effect, but also probably by the has-enough-disposable-income-to-waste-on-alcohol effect, the has-enough-friends-to-go-out-drinking-with effect, and the alcohol-industry-pays-for-the-research effect.
6. Imagine tobacco companies in 1970 found some small real benefit of smoking and hyped the crap out of that. Then it all gets confounded by doctors recommending light smoking to reduce the risk of obesity, and then people who follow doctors' advice start doing that, and the tiny benefit starts looking bigger. They could have actually done this by emphasizing the fact that smoking protects against obesity. I am suspicious that something very similar is going on with alcohol.
Correction: in the present day, iodine is artificially added to the feedstock of chickens and cattle. But historically, you couldn't get very much iodine from any land-based foods.
The nuclear-war probability forecast seems unfortunately low. Looking at the source, it looks like they tried to estimate the probability of a nuclear war as a result of this crisis, took that as a "current probability per year", and then took the one-month probability as 1/12th of that. The current crisis will almost certainly be either over, or stalemated, in much less than a year.
If that understanding is correct, then the truncated-aggregate of 0.067% per month really means "0.8% chance of Russia-NATO nuclear war as a result of this crisis", which, OK, is more plausible but most of that is going to be concentrated in 1-2 months.
Roughly, take the Metaculus prediction of Russia taking <3 major cities by June as the probability of Russia not being able to win a conventional war against Ukraine at a price Putin can afford. There's a possibility of Russia winning in a prolonged war despite their crappy logistics, but there's also the probability that they take 3 cities by June because they nuked or gassed them in April, so handwave those as roughly cancelling.
That's a 30% chance of Russia not winning a conventional war against Ukraine, which would be catastrophic for Putin. Possibly existentially catastrophic; note the 20% forecast of him not running Russia in a year. So, if he can't win a conventional war, 50% for him to try an WMD war (against Ukraine, not NATO). But only 20% for "WMD" meaning "nuclear". So, 0.3 x 0.5 x 0.2 = 3% for Putin to use tactical nuclear weapons against Ukraine sometime in this crisis.
That's going to put huge pressure on NATO to do something drastic, and put Russia on hair-trigger alert for the possibility of NATO doing something drastic at the same time, and that gets us into Cuban Missile Crisis territory. I think we can manage that with maybe an 80% probability of avoiding an actual Russia-NATO nuclear exchange, but that's still a 0.6% probability of "World War Three" coming out of this - close enough to EA's 0.8%.
The 82% for "we'll at least stop short of mass city-killing" and 75% for "if it does come to city-killing, we'll see it in time to run" seem about right.
Good news is, I think the 78% fatalities in London (or Berkeley or any other major city) is on the high side, probably by a factor of two or so. This isn't the Cold War, with tens of thousands of strategic warheads ready to go. I don't think he can spare more than 6-10 warheads in the 100-500 kT range for London, with everything else he'll have to deal with; NukeMap seems to be broken right now but think ~40% fatalities rather than ~80%. Picking the low-hanging fruit in the densely populated urban areas rather than trying to flatten every suburb while simultaneously digging out every deep reinforced-concrete basement.
So, BOTE, six times the near-term monthly risk of World War Three and half the expected casualties if we do get that, or 72 micromorts.
Doubled if you do live in a densely-populated urban area, and quadrupled on top of that if you aren't prepared to evacuate on either very short notice or very modest levels of escalation. So do keep your NorCal friend's number on speed-dial.
Not taking three major cities does not necessarily imply that war continues until June while Russia is unable to take them. That would be imho deep under 30 %, given that from those cities, Russia had already taken one (Kherson), and second seems in imminent danger of falling (Mariupol).
Another situation when this question resolves negatively is if there is an armistice before those cities fall.
Why would Russia nuking NATO or NATO nuking Russia not involve nuking cities? I'm not seeing them using nukes short of general war and in a general war strategic bombing is pretty useful.
The fatalities swing around pretty wildly depending on what counts as dying to nuclear war and what you think is going to happen following a nuclear war; there'd plausibly be a lot of deaths from "didn't Duck and Cover, got burns/glass cuts, because everyone else also failed at this and the hospitals got EMPed/somewhat damaged there's no medical capacity so died of tetanus/gangrene" and "the water supply broke, died of dehydration". That said, if we're conditioning this on being well-informed and at least mildly prepared, those mostly go away.
Nuking cities per se is generally a dumb strategy, militarily speaking, and quite possibly politically as well. Cities are soft targets, but also huge, and by themselves pose only a diffuse and non-immediate military threat. So normally your most important targets are enemy weapons installations -- including of course ICBM silos, but also command centers, radars, submarines that might be in their pens, and airbases that might have strategic bombers (or air-superiority fighters) on them. You might spare some for the seat of government, if you have leftovers.
To the extent the military targets of importance are located *in* big cities, the cities get hit, too, but in perhaps a surprising number of cases, they aren't. In the US all the big ICBM fields are a long way from any cities of consequences. Offut is near Omaha, but Omaha isn't very big. Submarines are based in Kiings Bay, which is somewhat near Jacksonville, and Bangor, which is quite close to Seattle. Barksdale is kind of out in the middle of Louisiana.
Hmm, seems I'd underestimated the amount of stockpiled ammunition against world war.
It is of course not a coincidence that the United States during the Cold War located key strategic targets as far from big cities as possible.
> Along with reassuring me I made the right choice not to run and hide...
Sorry, I'm confused: how come ? The table above this quote shows that there's a 0.8% chance/year of global thermonuclear war -- an even which will likely wipe out our technological civilization, if not for good, then for a long time. This probability is way higher than most other X-risk estimates, including those from AI and asteroid strikes. I am kind of tempted to run and hide, if not immediately, then at least soon -- why aren't you ?
Well, the same table predicts that if there _is_ a nuclear war then there's an 82% chance it won't destroy London, and it's hard to see the end of technological civilisation if London is still standing.
And London is one of the most likely cities to be hit, so I assume that the 82% contains a significant chance of a nuclear exchange that doesn't escalate to destroying cities.
Even if London and Moscow and New York are destroyed, nobody has enough spare nukes to start targeting random cities in non-nuclear countries far from the action. Nobody is going to be nuking Santiago, or Kuala Lumpur, or Auckland.
Since the "fallout kills everybody" and "nuclear winter kills everybody" scenarios no longer seem to be taken seriously, I think technological civilisation would survive any nuclear war.
"Run and hide" is an appropriate response for something that hurts people in cities, but is much more manageable for people in rural areas. To the extent that the worry is "wipe out our technological civilization", that seems less likely to make "run and hide" a relevant response.
Well, there's a big difference between "living in a cabin in the woods" and "incinerated", so to that extent running and hiding still makes sense...
Depressing that Putin is now *more* likely to remain president. What's up with that?
Presumably the thought is that the first week of unexpected Ukrainian resistance is the most likely specific moment for an internal coup. If he has lasted this long, then there's still risk for him, but it's probably lower and only slowly growing.
Is there a way to sell on Scott's manifold biodeterminist thing? I bought a few of the options but it would be easier if I could directly sell one or two that are (in my opinion) overpriced.
Also, Scott, do you want me to predict what I think you will think as opposed to what I think is *true*? Clearly the market incentivizes me to do the former, but I don't mind doing the latter and losing a little bit of fake money for the sake of contributing to the knowledge pool.
That is to say, if I think intervention X is very important but I also think that you don't think so, then it would be irrational for me to buy X on manifold if I'm trying to maximize returns, but it would still be rational to buy X if I want to encourage you (or others) look into X properly. Is this type of reasoning encouraged or discouraged?
20 % chance on Polymarket that Zelensky will stop being president by April 22 seems too high. I'd short that to something like 10 % if I could could be bothered to figure out how Polymarket works.
Ad WW3 on Metaculus, it was 14 % before February 24 compared to 21 % now. I don't have a good framework on what is the correct probability of WW3, but 7 point jump because of the invasion of Ukraine seems clearly excessive.
EDIT: Also more than 50 000 dead civilians in one city(!) on 12 % seems way to high.
I was surprised to see some commenters questioning the value of markets that give probabilities. We've had these in sports for centuries. The parimutuel odds tell you a lot about a horse race, and make it much more interesting. In football they use a point spread instead so the odds are always 50% but the event is tilted. Again, that's very useful information for a spectator. There are a few bookmakers out there who have to digest the odds in order to make a living, but for most people knowing the odds is part of being an informed spectator.
A reasonable summary of the forecasters vs. experts seems to be 'They're about the same but the experts have making coherent up-to-date predictions as a low priority so they half-ass it while the forecasters take making predictions seriously' Which makes a lot of sense if the forecasters are listening to the experts and translating what they're saying into coherent predictions.
The prediction market I really want to see is where the Ukranian war will be at the end of the year.
I think the point about the "natural 20s" is backwards. My model is, on any day, Russia wins if they roll a natural 20. Each day that they fail gives them one less chance to do so by April 1. Ukraine keeps rolling a natural 1-19, which is exactly what we should think is most likely on any particular day, but it does give us a little bit more information.
> Will Zelinskyy no longer be President of Ukraine on 4/22?: 63% —→20%
It's painful to see his name written like that
> 5: Related: I’m testing Manifold as a knowledge-generation device.
I raised the question in an Open Thread, but are there any successful examples of them being used as a "future-generation" device? Given big enough volumes, prediction markets are open to "insider trading", where you can make use of hidden information (e.g. something that didn't happen yet, but what you're premeditating).
I'm glad to see the edit in the first section. This is a really common pattern in a prediction market with an end date and default outcome. Each day that the thing doesn't happen, the probability will shift closer to 100% chance of the default outcome barring a major event (which will often lead to it dropping to 0%). Things can change as events occur but if you wake me up on March 31st and say "Kyiv hasn't fallen. What are chances it holds until April 1st?" obviously I'm going to assign a higher probability than I would on March 16th, which is higher in turn than March 1st. But I still won't say 100% without new information.
On second thought, you know why this post is so weird? Because Risk! If Russia (well, Ural) invades Ukraine and drops to a 40% chance of winning (say, ~20 attackers to ~25 defenders), the likeliest outcome is that the probability monotonically drops towards 0 (well, not quite monotonically, but with fluctuations very much like this one).
There are 8 instances of "risk" in the posts and another 9 in the comments and Risk literally put Ukraine on the map, and nobody pointed this out?
Jumped straight down the comments when I read "London is a hub for the effective altruism community". Is it?! Can someone tell me more? I just moved to London, and have always been quite sad that due to certain factors tying me to the UK I wouldn't be able to get involved as much as I'd like. How do I find out about/get involved in EA stuff in London?
So I decided to try writing one of these markets ... https://manifold.markets/AlexPower/will-the-nytimes-wordle-for-march-2
And I discovered an interesting slug behavior in re-issuing the market for Wednesday: https://manifold.markets/AlexPower/will-the-nytimes-wordle-for-march-2-e311366c1028
One possible rational explanation for probabilities falling so consistently is that the distribution of expected evidence is very skewed. To see why, consider a prediction market for "will aliens appear by 2050?". What would the price of this event look like? Realistically, it would start off at some number X, and then slowly decrease in a linear fashion: in 2030 it would be X * 2/3, in 2040 X * 1/3 and finally converge to zero by 2050. The only possible alternative is if in some year, aliens actually do come (or strong evidence of aliens appears), and the number shoots up really high.
The Russian invasion isn't _exactly_ like that, but there are many aspects of it that lean in that direction. The military situation now is roughly "stable", and every day that it remains stable is good for Ukraine. But there's always the possibility of some kind of significant breakthrough for Russia, and that could happen suddenly (if they launch a major offensive, and that offensive succeeds). Of course, there is also the greater possibility of minor new info favorable to Russia coming in, as well as Ukrainian breakthroughs (like the recent breakthrough at Mykolaiv: https://www.dailykos.com/stories/2022/3/17/2086493/-Ukraine-update-Ukraine-retakes-75-miles-of-Russian-held-territory-pushes-toward-Kherson ), which would make a "rational" probability trajectory look more Gaussian (and so if the probabilities update linearly despite these facts, that's evidence of irrational updating-too-slowly).
It may well be a combination of both of these effects.
This should be obvious, but prediction markets have an obvious fail mode for things like nuclear war.
To state the obvious, there is no logical reason to "bet" on nuclear war because even if you win, nobodys going to pay up.
https://smoothiex12.blogspot.com/2022/03/well-well-well.html how about including some very interesting endnotes from William Engdahl's latest missive (he wrote Full Spectrum Dominance) and I consider him a foremost historian and truthteller all within one's own life prism and experience of course; check out the above blog on Ukraine and Russia with regard the biological weapons. FYI and IMO only. Also, I agree with some of the people on this "site" like in the absence of any meaningful information why not just GUESS? sort of Blinken.. just make it up and pray no one notices. :) have a great Easter week, maybe I will be back next week: I don't like hidden or international sites with codes in it: who knows what could REALLY BE GOING ON?.. :) cheers !