241 Comments
Comment deleted
Expand full comment
Aug 28, 2023·edited Aug 28, 2023

(1) "the biggest NO bets were superforecasters, people on the leaderboards, and rationalist celebrities"

Well I'm glad to be in such elevated company, but I was going "Nah" based on "it's never been done before and yeah yeah one of these days I know but not this time".

(2) I think prediction markets, as described in how they're behaving above, are working. Maybe not the way intended to be working by hopeful optimists who thought they'd all be about high-minded truth-seeking, but the way I imagined they'd work: when there's money to be made, people will concentrate on "how do I make the most money?" not "how do I answer this question best?"

(3) The political betting looks interesting; so people think DeSantis has shot his bolt and Ramaswamy isn't all that?

(4) I appreciate the use of Catholic iconography of the Sacred Heart to represent Christ, not any Protestant version (accept no imitations!) 😀 August 31st isn't a particular feast day of any note; if they put it for the Feast of Christ the King (last Sunday of Ordinary Time) then I might wonder if somebody knew something.

https://en.wikipedia.org/wiki/Feast_of_Christ_the_King

Yeah yeah one of these days... 😁

Expand full comment

A story about predictions in flight times? There is a Seinfeld episode about that.

Expand full comment

The same Alexandros Marinos of Ivermectin debate fame?

Expand full comment

As someone who's spent years trading on prediction markets, I can strongly confirm that returns are mostly driven by who can take advantage of idiots the most quickly rather than who is perfectly calibrated and accurate. Programmers who build bots that either notify them or auto-trade have a huge edge over regular people who just check in once a day or once a week. Probably not too dissimilar to the stock market, where high frequency market makers earn tons of money despite having very little to say on what companies are actually worth.

Expand full comment

Building on your analysis, one lens to consider is meme stocks; retail investors are notoriously unsophisticated and so markets that are disproportionately meme-driven should be expected to be more volatile and inefficient.

So to the extent that a market corresponds with widespread public attention, and furthermore lots of unsophisticated investors opining based on news, then you might apply a “meme market” correction.

Of course, lots of liquidity and real money markets should reduce this effect, but Gamestop and other meme stocks show that even in public stock markets, concentrated attention from meme investors can distort a big market for some time.

I think your analysis of “who is investing” is valid here, wonder if there are other signals that would be useful like news article sentiment.

Expand full comment

> I was tempted to blame Manifold-specific factors, like the ability to get starting mana instead of putting skin in the game. But real-money markets Polymarket and Kalshi got approximately the same results

I think prediction markets in general got a huge surge of new interest on social media, and people who didn't really know how to use them signed up to lose $100.

The only one I was following closely was Manifold, but the comments on the market were bizarre. I saw at least two arguments about whether the idea of a probability being assigned to the outcome even made sense! I have no idea what exactly the people on the "against" side of that argument think is going on in a prediction market. There were also a ton of r/wallstreetbets style comments about things like "whales trying to keep the market down" or whatever. Definitely a bit of a September moment for prediction markets.

Expand full comment

LK-99: Isn't this almost a degenerate market to begin with? A doomsday bet is something like "There will be a global extinction event before Jan 1, 2024 with >99.9% of humans dead". On such a market you would take a loan and go all-in against. If you win, free money. If you lose, nobody will be able to collect anyway.

The reverse scenario ("utopia bet" maybe?) then would be the LK-99 situation: room-temperature superconductors sparks the next industrial revolution, big money invests in real industries instead of hoarding or playing the finance roulette, everybody wins, losing 10k by betting against is probably a minor inconvenience. Otherwise, things stay the same except free money.

Disclaimer: I do not participate in prediction markets.

Expand full comment

> I do hope this situation will improve over time, as over-eager forecasters get burned and dollars flow from dumb money to smarter.

As we learned from gamestop, NFTs, and crypto, a market that's inevitably going to collapse can stay up a long time with people hoping to make money off the next boost. I wouldn't expect liquidity to ever truly make this go away.

Expand full comment

> I’m struck by how little of the competition at the top was about predictive accuracy

This reminds me of the difference between cash and tournament poker. In tournament poker, the payouts jump in discrete amounts, whereas in cash games each chip is just a dollar or whatever. The result is that in tournament poker sometimes it's optimal to take lines which _would_ be negative EV in a cash game (and may be negative EV in units of chips), but are positive EV in terms of expected winnings.

Expand full comment
Aug 28, 2023·edited Aug 28, 2023

>1: Not a prediction market per se, but British bookies are offering bets on how much Donald Trump will weigh at his arraignment.

Should be "were" offering, since the bet has been closed. Trump has already been booked and weighed in, but the number (215 lbs) was so obviously below his actual weight that "BetOnline" has refunded anyone who placed an "over" bet.

https://twitter.com/DaveMasonBOL/status/1694897511637348395

Edit: Also, unless I'm missing something (which I may be), that line has a couple of other errors. The only mention of such a bet in the posted link is here:

>One bookmaker site, Antigua-based BetOnline, on Friday offered an over/under of 273.5 pounds for Trump's weight upon his surrender to authorities in Fulton County

Which makes the bookies Antiguan (who, while members of the Commonwealth, are technically not British?), and the resolution criteria of the bet is based on his booking (by police, as opposed to the aforementioned bookies), which comes before the arraignment.

Expand full comment

I am somewhat considering betting yes on the jesus one

Is there a 1/5000 chance it accidentally gets resolved to yes? Or maybe that some insane Christian Jacks up the price?

I feel like at $0.02 it might be a worthy bet

Expand full comment

This whole LK-99 saga was bizarre. I immediately knew chances of it being real were ze... well, we can't use zero, so say <0.0001%. I'm not a superconductivity specialist, I only know that the problem is extremely hard and the payoff is incredibly high. Therefore a chance that a small team out of nowhere made an astonishing leap forward to not "sub-Antactric temperature", not "January night in International Falls temperature", but BOILING WATER temperature superconductor was ridiculous.

Expand full comment

Insofar as someone is probably correct a lot and puts their money where their mouth is I consider that to be the opposite of name dropping. I’d really be interested in seeing a reputation layer added based on accuracy of past predictions.

Expand full comment

Perhaps a quibble, but I'm not sure how meaningful the term "existential risk expert" is. I mean, Ord is a guy with a lot of opinions about existential risk, exactly zero of which have been shown to be even broadly correct?

Expand full comment

"I think Newsom and maybe RFK are overpriced"

How can you not think Harris is overpriced? If I could buy and sell at the listed prices, I could set up a hedge that would guarantee a win - sell 1 "Yes" and 1 "No". I collect $1.25, and am guaranteed to pay $1 whether she wins or loses. Or does the big spread just mean that no one's trading?

Expand full comment

“ That means if you’re Christian, you have to believe that one of these days there will be a version of this that resolves positive.”

‘None shall know the day or hour…’, right?

As long as people keep betting on “yes” with conviction then maybe not.

Expand full comment
Aug 28, 2023·edited Aug 28, 2023

"Second, on Manifold, the biggest NO bets were superforecasters, people on the leaderboards, and rationalist celebrities; the biggest YES bets were randos with none of those qualifications."

Does Manifold have a superforecaster index fund? :-)

( One admission: During the initial coverage (before the first report of fraud came out), I was guessing a bit less than 50% odds. I never placed any related bets. My rationalization is that the authors of the preprint, at least, were persuaded by what they thought they saw, so I wasn't guessing a strong no from the outset as a result. )

Expand full comment

Seems like a bit of a hazard to bet on flight delays. They never did find out who flew a drone at Gatwick, or if there even was one...

Expand full comment

How does the Jesus bet work. You bet $1, and get back $1 if he doesn’t return, winning $0?

Expand full comment

Alright, someone had to ask: what exactly was the "Goblin Mode" portfolio about?

Expand full comment

> Polymarket Second Coming of Jesus Question

This is why I have a hard time taking these markets even half-seriously except for Kalshi and the old IEM. Here the question is a joke in two ways when it should only be one. A low-effort way would have been to base resolution on announcements from the Catholic Pope. The "serious" questions are way too often a joke in this way too. Question ill-defined, resolution unclear.

Expand full comment

The superconductor prediction market analysis is lacking

Specifically

- the price spike after some paper came out with theoretical support for LK-99. The ability of the market to digest this information is uncertain.

- The spike also coincide with spike in American superconductor company stock price. So this seems like a whole market thing, not a prediction market failure.

- When the price spiked a minority [not sure how many but at least 2] Super-forecaster [identified by their track record and the tick icon] hold a big [>20k mana] yes position .

- Eliezer Yudkowsky initially on NO switch to YES on that price spike then switch to NO again. In total he gain 1% mana profit on this market, approximately 0 contributing prediction value.

- Zvi only gain 9% mana profit on this market which is not nothing but not much of contributing prediction value.

So my point is you can't say: "look at all these smart/rational people correctly predicting." at the time, a significant [?how many?] number of allegedly smart people already made incorrect prediction.

I mean you can say that *most* people with good predicting track record made the correct prediction this time round. But this is a much weaker claim.

And I am highly disappointed by the performance of Eliezer and Zvi, especially as I've heard claim that Rationality was supposed to help us make better prediction in novel situation, which I presume this is the case.

Expand full comment

Well done, this post has successfully shamed us over at manifold into buying the market down to 7% 😅

https://cdn.discordapp.com/attachments/1144040519765332089/1145881574601138176/image.png

Expand full comment
Aug 29, 2023·edited Aug 29, 2023

>>either a prediction market will be good, or you could make lots of free money

Is this even true? I've never worked with prediction markets before but I did look into it for LK99 (since I was pretty certain the markets were way overvalued), and as far as I could tell between legal and other frictions, for an American citizen resident in America, I could only have made a small amount of actual dollars, by tying up capital for a substantial time, and even that would have required me to invest a large amount of time (and run the risk of losing my money despite being right on the object level question because I didn't understand how e.g. Kalshi worked).

Stock market levels of `if you can spot a market mispricing you can make a killing' certainly did not seem to apply.

Expand full comment

> Also: “The resolution source for this market will be a consensus of credible sources.”

In other words, they're admitting that they can't resolve that market and they're not going to pretend that it might ever resolve Yes.

Note that this means No is the only possible side to bet on regardless of whether you believe Christ will return by August 31, handily explaining the current pricing.

Expand full comment

Manifold said on Twitter that they had tons of new traders who swamped the expert traders and pushed the price up a ton. The best traders were much more skeptical and the new sign-ups bet the whole farm on Yes.

Expand full comment

> RIP Yevgeny

This seems to have been something of a self-inflicted wound? Why was he on a plane flying out of Moscow? Wasn't he supposed to be in Belarus?

Expand full comment

>This might be useful for travelers, but how does Wingman itself make money?

One possibility is "travellers who would lose a lot of money if late bet Yes in order to hedge against that; Wingman is taking the other side of that and therefore makes money as a flight-delay-insurance broker".

Expand full comment

> I claim they were dumb.

I know this is an obvious point, but it really would have held more weight if you said this before the crash. Of course you know you always thought it was dumb, but memories are fragile and it is very different from actually declaring it.

Expand full comment

Superconductor: There was a bias towards "exciting result" but I also suspect some Yes buyers had a kind of ideological bias towards the expert establishment being illegitimate and real science breakthroughs coming from maverick underdogs (undercatgirls?).

Biden and Trump are too low on that Polymarket, another example of a bias towards exciting results (barring someone having insider information about their health, not just what is talked about on cable news).

Wingman: In addition to the hedging that has been mentioned, this seems like a problem where people who think they have more information about a specific flight are often overconfident. There is plenty of data about flight delays and we have a pretty good idea of what variables cause them.

Expand full comment

I would bet on Kamala Harris as Team D nominee before RFK Jr.

The DNC would never allow it.

Expand full comment

Shouldn't people have been short on LK-99 for hedging reasons? There's both the xkcd 955 motivation and the fact that the kind of person who uses prediction makets is dispropotionately likely to work in an industry that would benefit from room temperature superconductors.

Expand full comment

Reposting this excellent article about the LK-99 saga (credit to beowulf888 in Open Thread 291):

https://www.nature.com/articles/d41586-023-02585-7

Expand full comment
Aug 30, 2023·edited Aug 30, 2023

Appreciate the write up on the tournament. I was wondering why I got a bunch of random twitter followers over the past 24 hours. Just to share some thoughts on the tournament of my own

1) I didn't play the tournament at all like the 2nd place guy. For basically any prediction market I've ever been in, I've totally ignored the fees and have just tried to get the answer right. I had no idea that some knew how much money everyone had the entire time. The tournament rules were changed to specifically not allow this, but they were able to do it through the API anyway. I didn't care at all when the markets were ended and I pretty much avoided bonding markets that were ending soon until the last month of the tournament. This was probably stupid in hindsight, but I treated this like any other prediction market and I've found that bonding is for losers unless you're really really really disciplined. Finally, I only did the bot monitoring that Robert talks about because I kept noticing these insanely bad trades would happen frequently and my main competition at the time (Johnny 10 Numbers) would then buy the market back to fair value within an hour. I assumed first place was the most important factor in getting the money, so I don't think I had a choice

2) Predictive ability was important. While I did make some fake money reacting to breaking news, I only lost money in 11 out of the 48 markets that I participated in. Most of the markets where I made money reacting to breaking news (will PI survive, basically all of the Trump indictments, many elections), I was on the right side anyway.

3) Competition was soft. Not trying to be mean, but there just weren't enough people playing after the midterms. Markets that would have moved on PI in a few minutes on news took twice as long in this tournament

4) They really dropped the ball the last month. There weren't any new markets put up for the last month plus of the contest. The only one that was remotely in play was Trump getting indicted a third time. The rest of them were just bonds. This meant that the contest turned into a who can monitor the webpage the best and wait for someone to make a stupid bet to buy the other side. As the person who was likely to win, this sucked.

Expand full comment

"NinthCause and SG are Manifold co-founders."

This should concern anyone who understands anything about conflict of interest and money.

Expand full comment
Aug 30, 2023·edited Aug 30, 2023

"I do hope this situation will improve over time, as over-eager forecasters get burned and dollars flow from dumb money to smarter."

Is there an upper limit of dumb money? Each new event-to-predict is a new opportunity for a new hype wave. There is plenty of various levels of dumb in the stock exchange and other markets (which are very well saturated having been around centuries), and it not like world has run out of stock - housing - crypto - (next thing) bubbles.

Expand full comment

Getting the Jesus market to $0.02 is in some ways less impressive than getting the clinton and pence contracts to under $0.04. The Jesus market is only one month, while the primary resolves in almost a year, and you can get a a high yield savings account with a yearly return of 4.75%. This is one of the nice things about fake money markets, readily interpretable real money markets were a ZIRP.

Expand full comment

The superconductor forecasting failure is pretty troubling to me. Consider that the overall users of prediction markets are highly likely to be in the adjacency. When the adjacency cannot gather correct information on an easy-to-interpret (within, I'd say, the first 3-4 days) instance of scientific hype, it speaks to two things. One- the adjacency is eager to bet on things even when they have no way of correctly assessing priors. Two- there's a serious deficit of applied physicists, chemists, and materials scientists in the adjacency. Considering how many thermodynamics-based claims are made and discussed here (the obvious one is AGI takeoff without regards to heat transfer or how little variance in the universe would be explained by new physics) it's kind of stunning. I'm glad that at least a few superforecasters and big names got it right, but that's more likely due to it fitting the "shape" of a poor claim rather than any assessment of the data or field knowledge on their part.

You don't have to be an accredited expert to have deep knowledge of these fields, either. You just have to know math and read and talk to people a lot in order to develop the heuristics that allow you to assess a matsci claim as non-credible without just going "well, it sounds like non-credible claims in other fields." A few years back, I saw a list of recommended textbooks in a variety of fields on LW. At the time, there were none on non-theoretical physics, chemistry, or materials science. Perhaps things have changed since then, but I can't help but think that the rationalist community is somewhat, to use a popular expression, "spending a month in the library for things that could be discovered in a day in the lab." Empiricism isn't everything, but evaluating empirical results can sometimes only be done with knowledge of empirical methods. How many people here knew what small-angle X-ray scattering was before the superconductor fiasco?

Expand full comment

Hi. My name is Bayram, I am the co-founder of Wingman. Thank you for mentioning Wingman, happy to answer your questions:

1) Right now we are charging 0.5% for each transaction.

2) But, you are absolutely right, our vision is to help airlines & logistics companies hedge their bets. I have shared more in this post here: https://t.me/web3_growth/10

Please feel free to contact me if you have more questions

Expand full comment