Mantic Monday 7/31/23: Room Temperature Superforecaster
Plus more CFTC, X, and UFOs
Surely Nobody’s Going To Rig A US Election To Make A Buck
Kalshi is a legal prediction market trying to comply with regulations. They asked their regulator, the CFTC, for permission to make prediction markets about the upcoming elections. In grand regulatory tradition, CFTC has drawn this out into an excruciating yearlong process that has aggravated everyone involved.
Last month was the Comments Stage, when the public gets a chance to submit comments on the pending decision. A total of 1380 comments were submitted, although 180 of those were by a person named Chris Greenwood who somehow submitted the same message 180 times. This is probably a metaphor for something.
Most comments seemed to be anti-Kalshi. Some big anti-market and anti-gambling organizations urged their audiences to participate, especially a group called Better Markets. Most of these people’s talking points involved gambling on elections being a threat to democracy: what if it incentivized people to rig elections?
Is this a realistic fear? There are already so many people who have very very very strong opinions about who should win US elections that adding some gamblers to the mix won’t matter much. The maximum bet a normal person (as opposed to a big Wall Street firm) can make is $250,000. There are already thousands of businesses and millions of individuals who have more than $250,000 riding on the outcome of US elections, just because politicians sometimes make laws that affect the economy. But also, have you seen people lately? People have so so many reasons to want to rig US elections.
Big Wall Street firms can bet more on Kalshi, up to $100 million, but big Wall Street firms already have hundreds of millions of dollars at stake in elections based on who passes the next Sarbanes-Oxley or whatever. In fact, the whole reason for Wall Street to gamble $100 million on an election is to hedge the risk that Bernie Sanders will get elected and cost them $100 million. Allowing election bets makes Wall Street less interested in elections, not more!
Britain already has legalized election betting. But British elections are still rigged by special interests and the media and the Lizardman Conspiracy and all the usual people who rig elections. Nobody worries about guys who have put $1000 on Labour at the local Paddy Power.
Still, many people had strong emotions about the possibility of gamblers destroying our democracy. I’ll mostly skip over these in favor of other more interesting comments, like:
Various Prestigious Economists. I appreciate these people. Every time the CFTC does this, they send in their comment. It’s always “We Are Various Prestigious Economists, And Have You Considered That Economics Tells Us That Prediction Markets Are Good?” Keep fighting the good fight.
Muhammed Wang. This is a bog-standard pro-market comment with nothing to distinguish it from hundreds of others. But there’s a statistical concept called the Muhammed Wang Fallacy: the incorrect inference that since the most common first name in the world is Muhammed, and the most common last name is Wang, the most common full name must be Muhammed Wang. It usually ends with “…but there is nobody with this name”. Apparently there is, and he supports prediction markets! Or it’s someone’s pseudonym, I guess.
Sinclair Chen. Sinclair works at Manifold; she can be spotted at most Bay Area ACX meetups. I didn’t realize the degree to which she goes hard: “CFTC, if you are reading this, know that there is blood on your hands.” This is not exactly the message I would have written. But I think, as the Catholics like to say, that it comes from a vice which is the excess or perversion of a divine virtue, and I appreciate her for being the sort of person who’s like this, sort of.
Andrew Robinson also goes hard, though I’m having a tougher time finding the divine virtue underneath this one. “The market is completely fraudulent and is completely controlled by the largest banks and hedge funds that act as the 23 direct market makers for the global economy. they have once again over leveraged every conceivable asset 100 times over, minimum, and the game is up once the balance sheet runoff explodes in july. this is exactly how entropy can be taught from now on, and these wavelengths of energy will reverberate through all of time.“
Aristotle Inc is the company that runs PredictIt. PredictIt competes with Kalshi; also, rumor says Kalshi turned regulators against PredictIt and gave them big legal problems. But PredictIt is also naturally interested in promoting permissive regulations for prediction markets, so I was curious what they would say. They not only support Kalshi, but urge the CFTC to allow lower minimum bets than Kalshi requested. This makes sense: Kalshi’s business model is closer to big businesses hedging risks, but PredictIt’s is closer to random individuals making fun bets.
Jacob Cohen describes himself as the president of his school’s forecasting club. I think we’re going to be all right.
Manifold Markets is sponsoring Manifest, an “inaugural forecasting & prediction market conference”, to be held at the Rose Garden Inn, Berkeley, California the weekend of September 22. Their website is short on details, but listed speakers and guests of honor are:
…now that I think about it I do remember vaguely agreeing to something like this, though I’m not currently planning to give any particular speeches. But Aella and Robert are great - and although I’ve never met the third guy, it seems appropriate for a conference called Manifest to feature someone named Destiny.
Manifold tends to do things on impulse and fill in the details later, so the schedule looks sparse. But usually the things they throw together last-minute end up being pretty good, so I’m looking forward to this.
Tickets cost $220, but can also be purchased with mana (Manifold Markets’ play money), at least until the CFTC notices. It looks like there’s an arbitrage you can use to get the tickets at a 10% discount - I think this is less likely to be a mistake than a preference to have people who can spot arbitrages 10% over-represented at the conference compared to everyone else.
Room Temperature Superforecaster
Maybe the long-awaited killer app for prediction markets is . . . debating superconductors?
First, the markets:
I’m heartened to see these two very big markets ($200,000+ volume, 2,000+ traders) within 1% of each other (as of time of writing). This is a really difficult question without an obvious prior, so the level of convergence suggests the markets really are doing their job…
…but Metaculus is much lower, probably because the other two are asking if any replication will be positive, and Metaculus is asking if the first replication attempt will be. It’s bad news that these numbers are so different, and suggests a high chance that this stays confusing and comes down to finicky resolution criteria.
Still, this has gotten lots of people checking the prediction markets, including Paul Graham:
…and around 500 others, according to the Manifold Active Users graph (source):
Aside from headline numbers, I’ve also appreciated prediction market comment sections as a good place to stay up to date on the latest developments (including a link to this thread)
Elsewhere In Forecasting
A blind mystic who allegedly predicted 9/11 is said to have foreseen a nuclear disaster that will ravage Earth before the end of 2023.
Baba Vanga, a blind Bulgarian woman, is rumored to have predicted some of the biggest events in world history.
She died more than a quarter of a century ago, but many of her predictions are said to have come true long after her death.
Now, her followers claim that Baba Vanga foresaw a devastating nuclear disaster that will unfold this year.
Big if true.
In what sense did she predict 9/11? Another article gives the exact text of the 1989 prediction:
“Horror, horror! The American brethren will fall after being attacked by the steel birds. The wolves will be howling in a bush, and innocent blood will be gushing.”
This is a 1989 prediction! If you’re calling airplanes “steel birds” in 1989, you’re just hoping that people forget you lived when airplanes already existed and then get impressed with you for predicting them. Come on!
(you could argue that the second half is about Assistant Secretary of State John Wolf and Deputy Secretary of Defense Paul Wolfowitz howling for war with Iraq from within the Bush administration, but Ass. Sec Wolf played a minimal role in the war buildup so I think if you are being very strict in your interpretation there was really only one wolf involved.)
Anyway, Vanga’s other predictions for 2023 include:
Earth’s orbit will change
A powerful solar storm
“The end of natural pregnancies . . . all babies will be grown in laboratories, with states and medical experts deciding who gets one.”
Again, big if true.
PredictIt Gets Another Stay Of Execution
In 2014, researchers at a New Zealand university created PredictIt, a real-money prediction market focusing on US politics.
Real-money prediction markets are somewhere between unregulated futures exchanges and gambling. The US restricts both these things, so it restricts prediction markets too. PredictIt asked the CFTC, the relevant regulatory body, to let them operate anyway, arguing that they were academically valuable and would limit bets to relatively small amounts of money. The CFTC agreed and granted them a “no action letter”, a not-really-binding commitment agreeing not to bother them as long as they followed certain rules.
In 2022, Kalshi, a more savvy prediction market with more friends in government, applied to be a fully-regulated futures exchange. Either because of direct action from Kalshi to crush competitors, or just to tie up loose ends, the CFTC revoked their no-action letter against PredictIt and told them to shut down. PredictIt sued the CFTC, saying their decision was “arbitrary and capricious” and violated federal regulations saying agencies had to explain their actions and give people a chance to respond.
The original court that was hearing the lawsuit dragged its feet, so PredictIt appealed to the Fifth Circuit Court of Appeals, who issued a preliminary injunction allowing them to keep operating. Now the Appeals Court rules in their favor (article, court opinion), accepting most of the legal philosophy behind their challenge, like:
That no-action letters are real federal regulations, and agency actions overturning them must meet the normal standards for agency actions
That CFTC didn’t meet these standards in their original letter
In response to the lawsuit, the CFTC rescinded their original cancellation of the no-action letter, then cancelled it a second time in a way that put more effort into complying with the regulations around agency action. But the Appeals Court ruled that they were going to rule on CFTC’s original bad action , and not this later better action, and let other courts figure out what to do about the better action.
I don’t know whether this means things are looking good for PredictIt, or whether this means the CFTC will start a new case with its better action and the court will agree that it is better. The decision didn’t seem to move the market on overall lawsuit success.
Although I like the result of this decision, I’m worried about the ruling that no-action letters constitute binding commitments whose amendment or cancellation requires careful agency action with every t crossed and every i dotted. Why would any government agency ever give a no-action letter now?
The Mantis Of Wall Street
I ran into some finance people at the NYC meetup this week and asked why they weren’t using more advanced forecasting technology - prediction markets, superforecaster tournaments, calibration training, that kind of thing. A common reply was “who says we aren’t using it?” When I asked for details, the two types of answers I got were:
We’re using in-house proprietary software, we won’t tell you anything about it, and even if we did, we’ve optimized for making it hard to explain so that any accidental leaks don’t risk threatening our competitive edge.
We use Metaculus! It’s great!
I asked someone if they were doing some kind of formal corporate partnership deal with Metaculus and they said they didn’t know it was an option. Seems like a potential opportunity!
Fatebook (site, explanatory post) is a site by Adam Binks of Sage, intended to make it easy to make your own predictions (including about your personal life). Then you can track your calibration and Brier score:
Fatebook is pretty similar to the old PredictionBook.io website, but the PredictionBook team says they’re getting to the end of their ability to maintain the site and that Fatebook is a worthy successor. There’s a function to import your PredictionBook history onto Fatebook.
Also check out The Base Rate Times, a prediction-market-based news panel thing:
This Month In The Markets
Metaculus considers UFOs:
This covers literal UFOs, SETI, and Avi Loeb’s work trying to recover anomalous meteorites, plus any other way aliens might make themselves known to us in the next 27 years.
SEC vs. Coinbase. I was surprised the Ripple ruling didn’t move the market more.
Sorry for two crypto markets in a row, but it seems important that an exchange with tens of billions of dollars on it has 41% chance of collapsing in the next few years.
Looks like the most recent round of Twitter changes really hurt.
Some evidence that Russia’s position in Ukraine is looking better than expected earlier this year
I thought they had already passed this bill, but they’ve only passed part of it. You can see the details in the comments.
The rise and fall of Threads.
1: The Existential Risk Persuasion Tournament (see here) is releasing some more information and some corollaries of their outcomes - see for example Who’s Right About Inputs Into The Bio Anchors Model? I’ll combine some of these with a Highlights From The Comments post once they get all of them out.
2: A story about how insider trading on a prediction market almost leaked the big AI risk statement before it was supposed to become public.
3: At an ACX meetup, I was asked why journalists don’t cite prediction markets more often, ie “The new paper says there is a room temperature superconductor, but experts are skeptical, and the forecasting engine Metaculus says there is only a 10% chance it will replicate”. I didn’t have a good answer; do any journalists have an opinion on this?