deletedFeb 14, 2022·edited Feb 14, 2022
Comment deleted
Expand full comment

There are quite a few problems with a prediction market here (real-money or not):

1) certainly both the US and Russia would spend $100k to influence these markets if the top-line number were actually something important. if the markets are giving accurate numbers now, it is only because nobody important cares.

2) the counterparties I want to bet against are the news media sources that are (IMO inaccurately) claiming a high risk of invasion this week, not John Q. Public who thinks a war is likely because they've been listening to CNN and think 35% is a low number.

3) the "Russia invades Ukraine by the end of the year" markets presumably will move if Russia does not invade Ukraine this month. However, as I think it's very possible Russia will invade later this year (and not at all possible this month), my priors won't be effected at all. I don't even know what I am supposed to do with my money to communicate that through the market.

My personal estimate is that "Russia invades Ukraine this week" is at most a 1% chance. PredictIt doesn't seem to have a market, and Manifold says "35% by the end of the month". Metaculus only has "65% by the end of the year".

There is a very limited amount I am willing to put money on this, though. A return of 1% (v. a loss of 100%) is too minimal compared to the risks of "Joe Biden really wants war", "Vladimir Putin is mind-controlled by hostile aliens", or "my math is off by a factor of ten".

Expand full comment

Shouldn't an extension of the cube be that people can make their own markets, and profit a tiny percentage off the volume? Would incentivize high quality questions.

Expand full comment

Reposting this from the OT. Scott - last week in your passage of polymarket post you said the following:

"In 2010, Philip Tetlock (one of the signatories on the pro-prediction market letter) did some pretty basic forecasting work, not even prediction market level, and proved that he could significantly outperform top analysts at the CIA with access to classified information. The government refused to hire him or use any of his methods, and continued shutting down new prediction markets as they arose."

This seems objectively false. Starting in 2010, the Director of National Intelligence (DNI)'s office of Incisive Analysis (OIA) at the Intelligence Advanced Research Projects Activity (IARPA) funded the Aggregative Contingent Estimation (ACE) Program. This wasn't quite prediction markets, although it did run contests based on predictions and reward those with the best predictions. This program also cites Tetlock's good judgement project as a resource and founding source of inspiration. In fact, Phillip Tetlock was a winner of one of the ACE prediction tournaments in 2013. I'm not sure to what extent this is going on, wikipedia makes it looks like the ACE program only lasted until 2015, but an interview with IARPA's new lead in 2015 makes it sound instead like they considered ACE a success and will be implementing it more broadly now.

Either way though, your initial claim of "The government refused to hire him or use any of his methods" doesn't hold up. They clearly did make use of his methods, and continued working with him for at least the next half decade or so.


Interview with Tetlock where he talks about how the GJP was linked to IARPA: https://blogs.scientificamerican.com/cross-check/can-we-improve-predictions-q-a-with-philip-superforecasting-tetlock/

Wikipedia article on ACE: https://en.wikipedia.org/wiki/Aggregative_Contingent_Estimation_(ACE)_Program

IRPA Description of ACE:


interview with IARPA director in 2015 where he talks about ACE:


Expand full comment

I think Manifold is off on the Ukraine question because people don't care much if they get their play money back. Probably a lot of people think of 'advocating that war will likely happen' as close to 'advocating that war should happen' and vice versa. There are a lot of people who would think of betting against war happening as a kind of advocacy, that you think war shouldn't happen. So long as it's play money, the actual question of 'will war happen' may not even cross your mind before your political hindbrain starts advocating.

Expand full comment

"I tried to spend some play money to correct it and it snapped back to just as wrong as it was before. I have no explanation."

I have an explanation:

You didn't spend enough to correct Manifold's entire population-level mismatch between war bulls and war bears. You only spent enough to momentarily adjust the quote to something that was out of step with the composition of users on Manifold, and someone snapped it back to the correct number *for Manifold*. Nobody can arbitrage this against other markets because it's fake money. Even on the real money markets, high fees often prevent arbitrage.

Expand full comment

Does that imply that Scott (or anyone else) thinks making a front end for Augur would be a particularly high impact opportunity for a developer?

Expand full comment

The analyses I've read say that the current week is critical. Russia's "military exercise" is only scheduled to last until Feb 20, and keeping their troops on alert beyond that will look ridiculously suspicious. Even aside from image, they can't *afford* to mobilize their troops for too long, it drains morale and wastes their fuel reserves. And if war breaks out, they want it to be during the winter, while Europe is reliant on their gas, not during spring, when Europe (especially Germany) could easily sanction them.

So I don't agree that the different timescales justify the different predictions. If they're going to invade, they're going to do it soon, not wait around and then suddenly do it in May.

Expand full comment

I guess I'm missing something about prediction markets. If I understand correctly, they're an attempt to harness the "wisdom of the crowd" by 1) maximizing the size of the crowd, 2) making sure the predictors are being serious by requiring them to put money on it.

OK, but what an earth makes them likely to be reliable about what's actually going to happen? It's not up to the market as to whether Russia actually invades Ukraine, and why should any number of uninvolved punters be any more correct about it than a smaller number of Russia experts?

Expand full comment
Feb 14, 2022·edited Feb 14, 2022

Could "using a Nobel Prize winner as a sperm donor" count as a genetic engineering technique that raises IQ points by more than 10 points on average? I suspect that you could get that up and running today and offer it profitably for less than 25% of median household income. If you combine that with embryo selection a 10 pt average effect seems plausible.

Expand full comment

Hey Scott, this is Alexander at Insight Prediction here, the new academic prediction market now with a public beta open to non-US residents. I'm a big fan of yours and so we certainly appreciate the link. One thing to note is that many of the current participants in our market and in our beta in general have made hundreds of thousands of dollars in prediction markets. This market has few participants, but make no mistake, these are sharks making large bets. That said, the sum of their judgement can still collectively be wrong, and it will be interesting to see if the market is right in this case.

Expand full comment

Manifold is at 48% because it asks "before February", not "before 2023", which is what most other sources ask. If you look at that question, then it jumps to 55%, which is still weirdly low. https://manifold.markets/NiclasKupper/will-russia-invade-ukrainian-territ

Expand full comment

I think there is a substantialy higher chance that Russia will take Odessa than Kiev. Metaculus on this seems misaligned.

Odessa is a Russian speaking city, close to Russian army in Moldova, with which invaders might want to link up. Kiev, on the other hand, is full of Ukrainian nationalists, an attempt to take it might lead to ugly things increasing the probality of severe sanctions on Russia.

Also, according to supposedly leaked Russian plans, invasion should start from the south (of course this might just be intentional misinformation). See here: https://www.youtube.com/watch?v=UNIU6TRsRzk.

Expand full comment

As I have an apartment in neighboring Berdyansk: Mariupol is NOT yet under control of the "separatist" - though I am sure that it is by far the most likely of the 4 to get invaded by Putin's fighters. (close to the border, small, nobody cares about it, on the way to Crimea, if Putin plans "only" to cut out enough territory for a land connection to Crimea. Which is also true for Berdyansk.) People are native speakers of Russian, and were often pro-Russia before the war (2014- ....). As they all watch Russian TV and can all see that there is zero truth in its narrative "Ukrainian Nazis about to invade Donbass - Putin-Patrol to the rescue!" - no longer so.

Expand full comment

Should I interpret this as people being super unconvinced that the US is willing to confront Russia directly unless they're invading a member of NATO?

Expand full comment

If you bet Cincinnati and got 3.5 you won. I think the Houston mattress salesman bet Cin on a money line, so he would have lost.

I think that Putin will invade Ukraine and take the whole enchilada. But that is just me and my years of studying Russian history, culture, and politics. FD, my mother was born in Ukraine and left at age 12 in the middle of the 1930s. She loathed both Ukrainians and Russians because they were Jew haters.

The most insightful argument for no invasion has been made by Holman Jenkins in WSJ:


FWIW. You pays your money and you gets your choice

Expand full comment

> I don’t know what possessed him to make a Joe Rogan Georgism interviewee market, unless he’s gunning for the position.

Mostly just curious how far the movement's reach will be in a year, and that's the largest podcast that I know of. If one is betting Georgism is on a similar trajectory to public saliency as UBI, we would expect it to start cropping up in places like this sooner or later (he's had plenty of guests on to talk about UBI over the past few years).

I also made a few other Georgism-related bets, such as whether the ongoing project in Richmond Virginia will take some concrete next steps, and whether my other homeland of Norway will see at least one major party adopt it as part of their political platform.

I also generally agree that Manifold's major issue is that I don't want to tie up my fake money for months or years just to push a bet's margin by a couple of points. Not sure what the solution is there.

I also made a bunch of bets about crypto games, related to my ongoing research for my day job. This bet on the price of Axie Infinity's $SLP token was a real photo finish:


Expand full comment

A problem with the strategy of making a market asking "will I be convinced of X" is that adversaries can buy a lot of NO (if X is true), then argue you into doubting X, so it is not a very symmetric truth-finding weapon.

Expand full comment

I spend tachyons on the https://www.metaculus.com/accounts/profile/119291/ track record - they performed worse than the community average on 4 of the 5 resolved questions they had predictions on. The only comments they made are on that question.

https://www.metaculus.com/accounts/profile/120816/ and https://www.metaculus.com/accounts/profile/122261/ are both also commented only on that question. And, I mean, just look at the comments they made!

That definitely causes some suspicion.

Expand full comment

The Lithuania question seems a shorthand for "all out NATO-russia war" to me. Russia would certainly invade Lithuania in that case - the Suwalki gap at least. And invasion of Lithuania would surely trigger a large scale war.

So it's not a crazy question. But 18% is way too much.

Expand full comment

Now that we have markets where users can pose questions, can we begin part II of the road to futarchy and start doing conditional predictions? E.g. US GDP is above $x in 2025 and a democrat is president vs US GDP is above $x in 2025 and the president is not a democrat.

Expand full comment

Can somebody explain to me why all these people think there's going to be a colony on Mars by 2050? Slime Mold Time Mold writes that Mars would be a great location for a charter city, which just seems obviously untrue. Wouldn't it be a million times cheaper to build something on Antarctica or international waters or like, buy an island from a developing country?

Expand full comment

Maybe I'm failing to parse something important, but Ideamarket feels... Incoherent?

Here's my attempt to summarize how it works:

1. We have two objects: *Users* and *URLs*.

2. A *URL* is a link to an article, tweet, picture, or other content containing a truth claim.

3. A *User* is someone who votes on whether or not a *URL* is accurate.

4. People can deposit money in a *URL* to make it appear higher in the rankings. This gives the *URL* owner dividends.

5. People can deposit money in a *User's* account. The more money the User has, the more influence their vote has. This gives the *User* dividends.

6. This vote appears to have no affect on the rankings. Or maybe it does? They don't bother to explain. My best guess is that the *URL* will have some kind of "Influence-adjusted percent of *Users* that think this is horseshit" indicator. The votes will affect this indicator.

From these conditions, Ideamarket extrapolates:

1. People will be incentivized to invest in *URLs* they agree with to keep them near the top.

Why is this the case? I'm not sure. They do some hand-waving, but there's no clear chain of logic. I'll go through some possibilities.

I. Investing for dividends

You don't appear to get any dividends for investing in a *URL,* only owning it. That takes this off the table.

II. Investing for ideological reasons

You could invest for ideological reasons. Suppose you really like a story Ann Coulter wrote. You invest the $12,000 it currently costs to make the top-10 on Ideamarket. Congratulations! Your story is now the 10th most visible item on a subsection of Some Website. In the real world, Google Ad impressions cost $2.80 per thousand views or $0.75 per click. For the same price, you could have made the Ann Coulter story a top result in 3,000,000 Google searches--or had 13,333 guaranteed to click-throughs.

And that's how much it costs *now,* with a userbase that's probably like, 1,000 people *at most*. If Ideamarket ever takes off, that's going to get exponentially more expensive. And let's assume it does. Would you rather be the 10th highest link on Some Website? Or would you rather reach 100 million people—nearly one-third of America—with a Superbowl ad?

III. Investing for speculation

That leaves investing in the hopes that other people invest, so that you can sell before they sell, but, like... for that logic to work, there has to be an impetus for investing in the first place? Unlike NFTs, you don't get a unique "thing" that only you own. Unlike cryptocurrencies, there's no early adopter incentive.

But let's assume people invest anyway. We've established it doesn't make ideological sense to invest. That means the only reason to invest is speculation. If that's the case, the highest ranking posts will be the ones that... speculators speculate other speculators will speculate. Probably that means the most popular newstory of the current moment, I guess. Who knows what totem speculators will latch onto?

Whatever it is will have little-to-nothing to do with truth, or even what people generally believe is true. Which means Ideamarket will be functionally useless, aside from making some people richer and some people poorer.

Expand full comment

> Improving adult IQ is really hard.

I once increased my IQ by 27 points (a little under 2 SD) in 2 days. How? I did an IQ test from a book. 2 days later I did another test from the same book. Because the questions were similar I was able to do a lot better.

I'm sure if I practiced doing lots of standard IQ tests, I would get better at them.

Expand full comment

Prediction markets for what some human will decide to do has the fundamental problem that that human can bet on his own decision.

Putin is by some accounts the richest man in the world, and would probably not spend effort insider trading on these tiny markets, but it still feels like a fundamental flaw of the whole concept.

Expand full comment

I can't image a less credible use for predictive markets than this.

There are ZERO people on these markets with any knowledge or skill in this area - unless Putin or Shoigu are participating.

Expand full comment

The Ukraine situation is completely exposing how utterly limited the information sources in my life are. It’s driving me nuts that I can’t understand this situation better. I’m floored by how low the prediction markets have the invasion percentage at, I would think it was near certain.

Expand full comment

"Based on the 97% YES rate, I’m guessing 13177 is in fact a prime number. "

I just plugged it in to check, and apparently the answer is "yes," to the question "can you just plug numbers up to this size into google and ask if they're prime and get a straightforward yes or no answer."

Expand full comment

I feel like the Valinor question is like the good twin of an assassination market - if someone is planning to clean up the backyard, they could make bank off that question. (Well, play-money bank.)

Expand full comment

I can't say I see a huge difference between people betting on whether an invasion of the Ukraine will take place and those who used to bet on how long a hanged man would twitch, back when hangings were public entertainment.

War is really ugly stuff. If there actually is an invasion, a whole lot of people will die -- many of them young, many of them children, quite a lot of them wholly blameless in any conceivable way -- and they will die in awful agonizing ways, by being shot and bleeding out, or being burned to death, crushed by falling masonry, having limbs blown off by bombs and shells, et cetera. Some people just have to figure the odds on that, because big decisions have to be made and that's their job. So be it. But for J. Random Well Off First World Internet Dweller to amuse himself by betting on death and suffering in far away Asia seems unpleasantly cold-hearted and ghoulish to me. I hope they all lose.

Expand full comment

I am less confused about some prediction markets begin at 20% vs 60% than Scott. A prediction market is only as good as its user base. If you would happen to have a prediction market with user base in Russia, then it would predict less than 5%, because it's simply not a topic in Russian media. If your user base is in Germany, you get a very different prediction than from a user base in France, or from the US, or from Easter Europe. They are just exposed to different media sources. Probably the same goes with a democrat vs republican user base.

In theory, these things even out once the user base is even moderately mixed. So if you have a few users from everywhere. But I would guess that the current markets still have user bases which are too homogeneous.

Expand full comment

The problem of bets taking a long time to resolve and thus not warranting the investment doesn't seem like a problem with play money- it seems like a problem with betting in any poor long term asset, such as cash. But if allow bets denominated in say, S&P500 index funds, or any other currency, then suddenly long bets don't have to compete with markets returns, which should allow profitable betting down to a much smaller inaccuracy.

Expand full comment

Your map is faulty. Mariupol is controlled by Ukraine.

Expand full comment

If you don't know anything (maximum uncertainty) shouldn't you predict 50% chance?

As disclaimer, I've been participating in GJP since 2013.

Expand full comment

Interestingly enough, while prediction markets were forecasting a greater-than-not likelihood of Russian invasion, financial markets were implicitly betting the opposite (both Russian and Ukrainian currency is trading stronger against the dollar, stable wheat and falling LNG futures) and based on the news this morning it seems like financial markets were right. I have no real analysis here, but would be interested in what other folks think of this divergence.

Expand full comment

Does the fact that people are now calling the capital of Ukraine 'Kyiv', rather than the English version of the name 'Kiev' have any bearing of the probability of an Russian invasion?

Expand full comment

Is there some guide/tutorial how one may go

- from money in bank account in Europe (Poland)

- to balance on prediction market

- back to actual money


Has anyone calculated costs of that? I am curious enough to try prediction markets, but so far ones where I have seen some opportunities turned out to be inaccessible/blatant scams/not allowing earning real money/hilariously expensive/etc.

Expand full comment

Are there any prediction markets that allow for the creation of private groups/tournaments?

Not impossible to manage privately of course, but having a polished front end and being able to piggy back on existing questions would be fun. Akin to setting up a private draft kings tournament instead of just participating in the public ones.

Expand full comment

Man, if Polymarket is easy to use I REALLY don't want to see the others.

Expand full comment

People with actual skin in the game (and probably better information than those participating in prediction markets) don't seem to be acting as if there's a significant chance of an invasion happening.

Zelensky and other Ukrainian officials have consistently downplayed the risk. Given that you'd expect the Ukrainian government to be the most motivated to frantically raise the alarms in the face of compelling evidence that an invasion is imminent, the fact that they're doing the opposite is, on its own, almost enough to convince me that there's a low chance of it happening. You can also bet that the US et al. have tried their best to convince the Ukrainian government that Russian troops are about to march into Kiev any second now.

Government warnings to leave Ukraine have been met with collective eyerolling by people who live there, government officials included.

Russians and Ukrainians themselves, by and large, don't think it'll happen.

Those who participate in prediction markets might be in the habit of giving too much weight to political theatre performed by Western governments.

Expand full comment
Feb 15, 2022·edited Feb 15, 2022

I don't think your analysis of the "By 2050 Genetic Engineering to Raise IQ" question is quite right. We could probably explain about 30-40% of the variance in European IQ scores just by offering UK biobank's existing IQ test to the people already in the biobank. There are other biobanks of similar size to UK that could administer similar tests and explain a similar percent of variance. If I run Gwern's code from "Embryo Selection for Intelligence", and set r^2 to .35, I get an expected gain of 10.2 points from selection from 12. No iterated embryo selection or anything else fancy required. Granted, selection from 12 would currently take 3-4 rounds of egg retrievals with today's loss rates, but I'd guess 15-30% of couples already go through that many rounds of retrievals during IVF today.

I actually think the much bigger issue here is cost. Median US income in 2019 was $31k. A quarter of that is $7500. There's not many places in the world where you can get both IVF and embryo testing on a sufficient number of embryos for that price. Maybe Greece or some country with single payer systems that subsidize the cost of IVF.

Otherwise, if we're talking US prices for egg retrieval, getting 10 IQ points of gain would probably cost on the order of $60k (assuming some biobank creates a dataset with roughly a million IQ-tested genotypes).

All that being said, 2050 is a very long time away and I would expect cost for a given trait gain to drop significantly well before then. There are multiple technologies, each of which could by itself get us across that 10 IQ point threshold, that seem very likely to be ready before 2050: In-vitro Oogenesis (and iterated embryo selection if you can get in-vitro spermatogenisis working), gamete selection, CRISPR germline editing, perhaps even whole genome synthesis.

There are reasons to do Oogenisis, gamete sequencing, and whole genome synthesis that have nothing to do with embryo selection, so I would be very surprised if none of those are somewhat cost-effective by 2050. So you either have to bet that it simply won't be affordable (possible but seems unlikely), or that it will be banned by every country (which just strikes me as kind of unlikely given that labs have been screening embryos for BRCA for like 10 years now, and polygenic disorders for 2).

Expand full comment

If a market allows real money, is easy to use, and allows you to create questions, what prevents sports betting? Is the idea that questions will be moderated?

Also wondering what scissor question will get people and moderators to disagree on whether it’s sports betting.

Maybe the thing to do would be to admit it’s gambling and fix the gambling regulations.

Expand full comment

Next time use a venn diagram!

Expand full comment

For question creation on manifold, are you better off nailing the exact percentage, or better off putting an initial bid in that's wildly incorrect, so that it attracts action? Is there a Cunningham's law here?

4% of your ante is less than your ante, of course, but maybe drawing in lively action could be useful?

Expand full comment

"Manifold is suffering from the usual play money problem, where if you only start out with $1000 in play money, nobody wants to lock it up for three years to make a 15% profit"

Shouldn't it be solved by ability to resell the "yes" and "no" tokens? As far as I understand, the prediction markets works in the following way: you buy one "yes" and one "no" token for e.g. 1 dollar, people can trade the tokens, and when the market is resolved you can get 1 dollar for one of those tokens, and the other is worthless. So, if people see that a "no" token is worth 0.85 dollars, while they estimate its current expected value as, say, 0.99, they can buy it now for 0.85, wait until it grows to 0.9 (and it grows to 0.9 because you are not the only one with such reasoning) and then sell it back. So you get some return on your investment much earlier than in three years, and even if this particular market is really stagnant, your money is not locked up because you can sell your "no" tokens at any point. This should be possible even if the money in question is play money. What am I missing here?

Expand full comment

See also https://forum.effectivealtruism.org/posts/HDoMrQFG76QtkdrZJ/impactful-forecasting-prize-for-forecast-writeups-on-curated

We are giving away $4,000 prizes for the best forecast writeups on curated impactful Metaculus questions.

Expand full comment

I feel like Lars' market equates to 'Someone who has the power to influence Joe Rogan's plans can make a lot by persuading him to interview me' and Vivek's market equates to 'Someone can make a lot of money by looking this thing up and telling it to me.'

Which brings me to the question, is subsidizing a market on the question 'I will get a blowjob in the next 2 days' meaningfully different from soliciting prostitution, and is subsidizing a market on 'my bitch wife will die in the next month' meaningfully different from soliciting assassination?

This is still my biggest noticeable confusion about prediction markets being implemented large-scale in real life, I don't see how you screen incentives to make good predictions off from incentives to actually change the outcome. Or if you don't do that, I don't see how that isn't very very dangerous, giving people huge incentives to create low-probability outcomes and collect a ton of money.

Or if we don't care about the danger, I don't understand how this doesn't shake out into the same anti-inductive reality as the stock market, where everyone has an incentive to change the outcome of a thing relative to their own bet, so what actually happens has more to do with people acting on those incentives than with the prior probability of the actual thing itself before the market was created.

Obviously if you have $50k in a market on whether Russia invades Ukraine, that's not enough money to incentivize anyone who's capable of meaningfully influencing that outcome to do so. But if the pool was $1B? Or if there's $50k on the market for something smaller and more tractable, like the outcome of a sports game where players are allowed to bid in the market?

Expand full comment

Wonder if it would be worth hybridising a PM with some social network functions, where you can create questions that only friends can see and bet on (otherwise hard to prevent a litany of personal questions). Probably best to have these self-adjudicate as long as the current state of the market is never shown. Coordination must be guarded against somehow though.

Expand full comment

One reason the Manifold market might not line up (that I don't recall you mentioning) is it specifically ends after February. An invasion on March 1st would still be a "no" for that particular (very short-term) market.

Expand full comment

Could you make a prediction market that satisfies all three criteria by making it in two pieces, with all the real money being handled separately?

Make a play money market that is easy to use and lets you create your own questions, then have it produce some kind of proof that you made the bets you made (e.g. by posting the bets publicly, signing each with a key connected to the play money account).

Using a separate legal entity, maybe in a non-US jurisdiction, create a pool that people can pay into that pays out according to how you prove you betted (e.g. the same way you prove that you own any other private key - for maximum ease, have the play money account offer a way to sign messages with the key, so that you can take a random string produced by the real money account, sign it, then have the real money account verify the signature, thereby avoiding the need for any security / crypto knowledge beyond what's required to use a bank account).

It seems plausible that there is some way to arrange the latter entity so that it doesn't count as gambling or a stock market. For example, you could arrange it as a not-for-profit entity that receives donations from people who value good forecasting and awards grants to people who have demonstrated good forecasting. You could instead have the real money entity assign people ownership of tokens that it never pays out on, but which can be sold to crypto-savvy users for real money, with the tokens being valuable because they can be used to claim crypto wallets created by the real money entity containing enough crypto to cover the cost of buying the tokens plus a fee. (That is, Mantic Manfred pays $100 into a central fund that uses it to buy USDT 100; Manfred proves that he made a bet for $100 at 50:50 odds on a play money prediction market and is awarded a token; the market resolves in Manfred's favour; Manfred sells his token for $190 to Crypto Chris; and Chris uses the token to claim a wallet containing USDT 200.)

Of course, I may simply have pushed the law-breaking bit into a complex enough arrangement that it's no longer obvious to me that it's illegal while remaining obviously illegal to a US lawyer or a clued-up crypto investor.

Expand full comment