I actually think we're really lucky that speculation in onion futures was banned like that. Without it we' only see the financial disturbances that speculators cause and not the ones they fix. With onions, though, we have this nice cautionary tale that stays relevant and is something important enough to be a test of the idea but not so important as to cause real problems.
If you're a carrot farmer, you can sell carrot futures six months in advance to the local carrot cake factory and know how much you'll get for them. Both the farmer and factory know what price they'll sell/buy the carrots for and can get on with farming/baking.
If you're an onion farmer, you have to grow the onions not knowing what price you'll get for them when they're harvested. You're exposed to the risk that onion prices will fall and for the owner of a bhaji factory, there's the risk that the price will rise suddenly.
And apparently the Onion Futures Act was the direct inspiration for trading in pork bellies and orange juice on the Chicago Mercantile Exchange – the onions were a big part of their business!
Huh, I've heard of "onion futures" being illegal before and didn't realise it was literally onions. I always assumed it was a type of future named metaphorically after onions - like futures on futures on futures on something - something with layers. TIL.
Another prediction market site that right now only appears to have two things, but I vaguely remember being pretty liquid for the Trump election: https://ftx.com/markets/prediction (I have never used it)
What do you think Metaculus would predict about its own demise? E.g. something like "predict the year in which Metaculus's monthly active users drops below 100 for 6 consecutive months." And how do you think this prediction effects people's other predictions? Do people care less about being right if they'll be right only after the site is no longer operating?
"Due to the nature of this question the best point optimizing prediction would be 99%, no matter of the real probability of Metaculus existing in 2030, as a non-existent Metaculus cannot make you lose points. Still, players are urged to predict in good faith."
Weakened versions of that note also apply to lots of other long-term questions too and will certainly play some role in how people predict.
On the AI-written book question - would a volume of short stories count? I would bet that GPT-4 could write a reasonably coherent short story, especially if you prune the more outre outputs .
Monkies at typewriters will eventually write a bestseller, *if* you have a competent human author/editor choosing which of their texts to try to publish, or which individual sentences/paragraphs to string together and in what order.
'AI will produce trillions of text strings and humans will find one that can be sold' is very different from 'AI will be told to write a best-seller and succeed on first try.'
I think the most straightforward explanation for why the Polymarket Trump contract is still trading at 6% is that 1) people who aren't familiar with crypto are put off by the complexity and inconvenience of getting their money into USDC so that they can trade there, and 2) people who are familiar with crypto have no interest in locking up their capital for two months for a 6% return, because there are much better opportunities for yield currently available in that space.
Is this true? 6% per two months is 40% per year. The amount of crypto knowledge it takes to invest in Polymarket is minimal - I think I could do it myself if the relayer was working. I don't think it's generally agreed that a rando with a tiny amount of crypto knowledge can make 40%/year risk-free-ish returns? (I think just buying and holding Bitcoin might plausibly be 40%/year, but not risk free).
How much are the fixed costs on it? At the amount of money you're betting, what is the actual expected return assuming there is in fact a negligible chance that Trump will be President on Mar 31?
I believe it costs 2% to enter and 2% to exit a position, so a 6% discrepancy is only exploitable for a 2% profit. These fees are paid to liquidity providers, not the platform.
No, I don't think someone with minimal crypto knowledge could necessarily do significantly better than 6% in two months. But I think the set of people who are familiar enough with crypto to invest in Polymarket, but not familiar enough to have any ideas for other low-risk strategies in crypto, is pretty small.
A really simple alternate strategy would be to hold Bitcoin and short the March futures contracts on a crypto exchange. Right now those contracts are trading roughly 4.2% above the current BTC price (46266 vs 44412, see https://ftx.com/trade/BTC-0326). So you'd have a "risk-free" return of 4.2% in roughly the same time that the Polymarket contract nets you 6%. Also, note that Polymarket is quoting 6%, but if you actually type a number in the box to buy, the "max return on investment" that it shows you is ~4.7% (and that number goes down as the size of your bet increases). So you can do basically as well as the Polymarket bet with an extremely simple futures strategy.
It's because people are paying for leverage. If you buy spot Bitcoin, you can only take a position that is as large as the amount of money you have in your account. On the other hand, if you buy futures contracts, you can take a position several times larger than your total balance. But because lots of people want to do this, they end up bidding the price of the futures above the value of Bitcoin. It's comparable to borrowing money to buy more spot Bitcoin and paying interest on the borrowed amount (and actually, there is a direct relationship between the futures contango and margin interest rates).
So is the apparently free money I would get by having+shorting Bitcoin to compensate for the risk I'm taking on that these people won't actually be able to pay me later?
Not exactly, because collateralization is handled by the exchange rather than the individuals who are "providing" leverage by shorting futures. The exchange will automatically liquidate traders' positions if their account balance falls too low, and will backstop losses that do occur if liquidations happen too slowly. Here is how FTX does it: https://help.ftx.com/hc/en-us/articles/360027668712-Liquidations.
So you are taking on risk, but the risk is not so much that the individual leveraged traders will go bankrupt. It's more like the risk that the exchange fails to manage its liquidation engine appropriately and also does not have enough capital in its insurance fund to cover a sudden cascade of late liquidations. There have been a few times when this has happened - the last instance of socialized losses that I remember was on Okex in 2018, see https://news.bitcoin.com/okex-socializes-loss-from-over-400-million-bet-among-btc-futures-traders/. But crypto exchanges have gotten pretty sophisticated over the past few years, and so I think this type of risk is minimal and not the main reason for the existence of the "free money".
That said, it's pretty common for US-based individuals to use those exchanges anyway - you just have to do so with a VPN so that your IP address is located elsewhere. FTX will let you sign up with just an email. The main downside is that if you don't provide photo ID when signing up, you have a maximum daily withdrawal limit of $9000. This may or may not be a problem, depending on how much you're interesting in trading and how patient you're willing to be when you decide to eventually withdraw your money.
Interesting; thanks for the information. I'm comfortable with the VPN and the withdrawal limit, but I'd be concerned about the risk of getting my account shut down. Do you have a sense of whether they can just cut off your access to the account if they find out that you're based in the US?
I'm hesitant to give you concrete recommendations here because I don't want to encourage you to do something that ends up losing you a lot of money. But I do know several people whose accounts have been flagged as belonging to US citizens. In each case the exchange's response was to require them to immediately liquidate their account and withdraw their funds, as opposed to confiscating those funds. I think the exchanges view this as a much better way of limiting their regulatory liability than holding on to users' assets. That said, this is merely anecdotal and I'm not sure that most exchanges have an official policy one way or the other.
You might consider creating a "burner" account with a minimal amount of capital on whatever exchange looks most interesting to you and then test what happens when you out yourself as a US citizen.
It's more like a 4% return per two months after you factor in the various fees, which would be about 27% a year. Second, it's not risk-free. I don't have any specific concerns about Polymarket (I've been betting through them for a few weeks now), but any kind of internet betting site has some built-in risks that don't exist with more traditional investments.
However, the most important point is that it's not something that you can easily repeat six times over the year to actually earn that 27% return. If I thought there would be similarly safe bets available all the time, I'd absolutely go to a lot of effort to capitalize on them. For a one time bet that will only net me 4% profit and expose my money to some level of risk, it's not such an easy choice.
Am I right that you're hypothesizing that right now Polymarket is unusually exploitable, and you're expecting most times not to have so clearly wrong a contract? If so, what makes you think this?
Polymarket just launched, so it's less liquid, and thus less of an efficient market.
Most of the "safe bet, decent return" markets are based on politics and basically exploit the irrationality of Trump supporters. I imagine this money will slowly drift away as Trump gets further and further from public life.
I actually think it's really just that single market that's unusually exploitable. Even the very similar "Will Biden be president on 3/1" market, will only get you about a 1.5% return, and that's a bet you actually have a chance of losing, so you shouldn't stake your entire bankroll on each bet. If you factor in your change of losing (let's say 0.3 of a percent chance that Biden is not president by 3/1), and you bet using the Kelly criterion, you're down to an expected return of 15% a year. That's still pretty good, but it's not outside the realm of investments that are available elsewhere if you're willing to accept some risk.
Also, I have to correct my earlier statement about the fees. Some of the fees are a one-time cost of getting the money into Polymarket, so they're largely irrelevant if you plan to bet repeatedly over the long term.
I think it would be a mistake to use annualized returns for prediction market forecasts. It is not likely that you can earn 6% on a bet this certain in a prediction market into perpetuity. The pricing on "sure things" over election night on PredictIt was about 5-7% (i.e. CA to vote Democrat), many of them settled within a week, does that mean the expected return on predictit was 1200% annualized? I mean, in one sense literally yes, but not in any useful sense. The return was a one-time return of 5-7%, enough to cover the exchange's fees, and some post-election horizon expected return that would be positive but vastly lower.
Likewise now, yes you can earn 6% in the next 2 months on Polymarket. I don't know what the risk associated with the platform/stablecoin is exactly, but its clearly the main risk you are taking. After those 2 months, will there be another 6% return in two months on something just as certain as this Trump market? Unlikely. So you have to do work on this crazy crypto market to get comfortable with it, put up enough money upfront to make that work worth your time, and take this concentrated platform risk all to earn <6% expected return as a one-time event, and considerably less on an expected basis over time.
In this case, you know there won't be - not for another two years at least - since it was predicated on the election. On top of that, it's not really a $20 bill on the sidewalk, but a $20 bill at the other end of a mudpit, but you have to put $300 into a box for a couple months, and there's a toll of 2/3 of the $20 bill to be allowed to cross the mudpit.
Well, it stretches the analogy a bit, but it's more accurate. You have to tie up your money to get the return, doing it is somewhat confusing, and - worst of all - the market (according to other comments) has a 2% deposit fee and a 2% withdrawal fee. Oh, and it's an online cryptocurrency site, so there's a nonzero chance that they'll just disappear with all of your money.
To push that market down, given that there are still crazy people putting money into yes, you need to risk a lot of money on a platform that hasn't been tested enough.
This is not quite accurate. The best low-risk DeFi yields are maybe in the 12-20% range.
I haven't maxed out the Trump one simply because I'm maxed out on the Biden one that resolves Mar 1st, and I want to keep capital liquid to take advantage of other markets that arise.
This is true if you are only looking at strategies that are very close to being "risk-free" (although see the futures arbitrage strategy I mentioned in my reply to Scott). But if you're willing to take on and manage a moderate amount of price risk, you can do a lot better than 12-20% APY.
Another way to put it is that any decent crypto hedge fund/prop shop would be very disappointed to return only 6% over the next two months, and they have capacity constraints that the average individual investor doesn't have. To Scott's point, these strategies obviously do require some level of sophistication and knowledge of the crypto sector, but my point was that I think most people who would consider trading on Polymarket in size probably clear that bar.
If you run a hedge fund, you capture ~20% of wins and are responsible for ~0% of losses.
You don't want to lose, of course. But your incentive to avoid small risks is a lot less than the incentive of someone putting in a million of their own hard-earned greenbacks.
The futures strategy is interesting but I suspect there's some margin risk with large swings? Have to look into it.
Sure, and I agree that hedge funds as a group are not well-incentivized to avoid tail risks. But I think most prop shops are also aiming higher than 6% by end of March, and for them it's their own capital at risk.
For the futures strategy, there is no margin risk if your short position is equal to your spot position and you hold both positions at the same exchange. The only risk is counterparty risk, i.e. the risk that the exchange you're trading on gets hacked/compromised/shut down. This is a legitimate concern, especially when your upside is just a few percent, but it also applies to trading on Polymarket.
The May/June 2020 upward deviation in predictions (Will AI progress surprise us? / Date first AGI is publically known) coincides with the release of GPT-3.
For "Date First AGI is publicly known" and similar:
It seems like this is a sort of problem where the time value of money really matters. I.e., a bet that yields $100 in 2025 is worth more than a bet that yields $100 in 2055 (even after adjusting for inflation), so there is the potential of having a huge thumb on the scale that encourages people to bet on sooner dates rather than later dates (or else encourages the "soon" predicters to bet and the "later" predicters not to).
How do metaculus or real-money sites account for this? Or do they even need to from a business perspective?
On Metaculus you win fake internet points and not money, so that's how they solve that problem. It's obviously both a pro and a con that real money isn't involved, but it works remarkably well considering.
Are you limited in the Metaculus Fake Internet Points you can wager? If not, that's a bit more comforting, but I think I would still be pushed toward betting on sooner outcomes because my discount rate for MFIP is very high.
You don't have to spend points make a prediction, a prediction wins you positive points if you're close to the mark and loses you points if you're wrong, but doesn't cost to play. So a discount rate shouldn't come into it, the only opportunity cost is on your time, not your points.
The downside of this is that "smart money" can't come in and correct a large error merely by staking more - all participants are weighted equally in the "community prediction" (actually they're weighted by how recent their prediction was).
Metaculus isn't technically a prediction "market" given you're not buying and selling these things - you're just winning points for correct predictions.
I'd like to see it gain more market-like properties, but it does work quite well in its current form.
That definitely makes things better than a dumb model of what a true prediction market does (I'm still interested in what they actually do), but I still think that there is an incentive to bet on things that pay out now. I.e., the only way that I'm going to spend any time on Metaculus over the long run is if I have a high point total that I can show off. If I bet on a bunch of stuff to happen in 2050, I'll never be around to enjoy it (maybe I'll be dead, but certainly I'll have quit Metaculus because none of my bets have paid off yet and I don't have any cool MFIP to show off). So regardless of when I think X will happen, I am incentivized to bet on them happening in the next few years.
Do they have payoff structures that mitigate this? Again, as a business model they may not need it, but it seems like something to think about if we want to draw conclusions about reality from Metaculus predictions.
the "community prediction" by definition weighs all participants the same. There is also the "metaculus prediction" which weighs users differently depending on track-record, though the exact algorithm is secret. However, the "Metaculus prediction" isn't visible until a question has closed. Most questions on Metaculus close prior to being resolved, so this is usually a decent chunk of time where you can no longer make predictions, and can see the "Metaculus prediction".
I think a question closing for predictions prior to resolution is designed to incentivise users to make an as-accurate-as-possible "final" prediction when there is still some uncertainty (you can update your prediction any time prior to a question closing). It's not a great mechanism IMHO and I think I'd prefer your reward to just be based on equal weight per unit time you're in each question without the "final" prediction being more heavily weighted. It's annoying that as an event gets closer the questions often close such that you no longer can use metaculus to see what the likely outcome is closer to the event.
I think the "metaculus prediction" is hidden prior to closing to prevent some kind of gaming - maybe they don't want users to simply copy the predictions of the most reputable users, perhaps this can lead to "bubbles" of false consensus, I'm not sure.
But yeah, I'd like to see it become more market-like such that we can a) view a prediction all the way up until an even occurs and b) allow heavier-hitters to move the prediction more.
You might think that this doesn't matter for Metaculus, because it's after all meaningless points, not money, but it's actually the reason I didn't bother with Metaculus after the first time I saw it. I can't be bothered with stupid internet points a decade from now.
Yeah. And since most people are going to think the same way I'd worry the outcome for long term predictions is just the same as if you got a group of people to submit probabilities, with no reward mechanism for correct results
Also, you have to assume that the traditional economy and the specific currency you'll be paid in would meaningfully survive a hard take-off scenario to begin with. Which I personally don't.
So, yeah, lots of room for considerations about what the money will mean in the event of the prediction going one way or another to take into account here.
Disclaimer: non-expert opinion; I got some USDC onto Polymarket last week and have read their docs and lurked their discord, but don't understand every detail of how it works.
Polymarket's transfer issues are recent, and the Trump-related markets have always been unreasonably pro-Trump (in my opinion).
The issue with the relayers is this: due to high trading volumes for ETH, it costs around $50 in gas fees to transfer USDC to Polymarket. Polymarket itself is picking up the tab for these fees, but they don't want to pay more than necessary. Therefore, they're only filling the gas tank when gas prices are relatively low, and they're not putting in enough for everyone who wants to process a transfer to do so.
Unfortunately, there is no queueing system, so your USDC will not be moved from the first relayer through to your Polymarket account until you click the button to do so when there is money in the gas tank.
There is no way to retrieve your USDC from the relayer except moving it through to your Polymarket account and then back out.
You can add money to the gas tank yourself to try to hurry things along, but it will cost at least $50 or so, and you should read up on the discord before doing so.
The Polymarket team is working on switching to a much cheaper transfer system, but it's unclear how long this will take.
Why does Trump to be President trade at 6%? The question has two parts: (1) why is anybody crazy enough to bet yes on that, and (2) why arent people eager enough to offer it at a lower price.
Subquestion 1 is really an anomaly, but one that clearly exists in any prediction market and in the real world.
Subquestion 2 has an easy answer -- Polymarket might not have the value-at-risk cap and trader cap that PredictIt has, but it does have some real risk of being shut down or declared illegal at any moment. PredictIt has a no action letter from the feds, Polymarket does not. Some combination of being a pain to get started with, being crypto-based, not having any assurance from US authorities that the site will not find itself in some state of illegality, and other operational risks mean that nobody is willing to put a large amount of money there without some minimum expected return to compensate for those risks.
As someone currently ranked in the top 50 on Metaculus having joined 6 months ago, and who has made >$100k on Polymarket having joined around 1.5 months ago, I have some thoughts.
First, on "But most of Metaculus is bread-and-butter questions about when such-and-such a system will meet such-and-such a benchmark, or how many parameters it will take, or things like that." Those questions are almost all from a specific contest run by the Open Philanthropy Foundation with $50k in prizes. See https://www.metaculus.com/ai-progress-tournament/
They do not actually represent the bulk of Metaculus questions, but it may appear that way if you sort in certain ways.
Second, Metaculus heavily rewards predicting on lots and lots of questions. If you just copy the community on every question, you gain net points in expectation, and do better if you can beat them, which you can do pretty easily just by looking at older questions where most predictors haven't updated recently. For two examples, see this thread https://twitter.com/avi_eisen/status/1357385286985121792. Often in practice, someone happens to see an older question and the community prediction looks wrong to them, they write a comment, which bumps up the question so a bunch of other people now update. Because of this dynamic, you need to be very careful if you're going to try to use Metaculus numbers as a prior/input into your model/etc. There are often fairly uninformed predictions simply because nobody has done a deep dive on the question for two months. I myself have predicted on 881 questions to date, and it's nearly impossible to update them all regularly. I try every other month or so and my brain just goes numb after scrolling through a few hundred and thinking which might have changed odds.
Third, as you can probably guess from my intro, it is absolutely worth it to figure out the kinks with Polymarket. The market is far from efficient. However, it is harder to beat than Metaculus, unless you're betting on the odds of a Trump coup.
On the order of 500k, which was greater than my liquid net worth. My personal finance is a bit complex right now, but let's just say that it was worth paying what would normally be considered high interest rates, given the risk levels here.
That is absolutely fascinating to me. Can you share more how you got started? Do you do something similar by profession (finance, statistics, data scraping)? With your skills, would you get paid more in a traditional job than if you were given 10k USD and told to go wild?
This is the closest thing I’ve seen to somebody literally been payed to be an oracle.
The bulk of the Poly money was from knowing in January that Joe Biden had won the election and was going to be inaugurated on the 20th, and being willing to take action to convert that knowledge plus cash into more cash.
I don't know if I have a long-term edge after that kind of stuff is factored out. I've never been too interested in a finance job because I felt the competition was too stiff and I prefer industries where I have an edge, which certainly applied to the political Poly markets. I've never had a traditional job.
Scott, if it interests you, the site Papers With code has a nice plot that shows the state of the art on these problems, and explains the datasets. I think you might find at least some of the problems to be exciting.
The Mini Imagenet is a subset of Imagenet, a dataset of a thousand classes of images. One shot learning is "learning to recognize X from a single image of X", which is pretty cool and non trivial thing (in my opinnion) for a neural network to do. It is described here:
"Presumably some AI-related advance short of AGI would cause an unprecedented world economic boom. I am pretty skeptical of this because there have been a lot of weird unprecedented things in economic history and they have never caused that severe a discontinuity."
We've never had one advance that affected every industry almost all at once; they've always had to diffuse out at the rate of human learning, human decision making processes and institutions, manufacturing and/or building infrastructure and logistics networks, and so on. That will still be somewhat true in a less-than-FOOM-takeoff, I think, but it doesn't mean there can't be a huge shock. Part of the value of AI is automating decision making, probably reducing those limitations. Wars and depressions do cause huge negative shocks, often followed by recovering faster than trend. In the Great Depression, US GDP fell by 30%. WWII was followed by a 30 year economic boom.
I will say I'm also quite confused about what I should expect to happen to prices and nominal GDP in a world where almost everything gets automated. My current understanding is that however indirectly, prices today are mostly about how fundamentally scarce the resources are to make something, and how much (and what kind of) labor goes into it. I expect asymptotically AI pushes the value of (non-human) labor to zero, and costs depend only on material composition and energy for production and maintenance, but the intermediate stages could be super weird in a slower takeoff depending on which labor components get zeroed out in what order and what prices of still-scarce-things we choose to bid up in response.
> My current understanding is that however indirectly, prices today are mostly about how fundamentally scarce the resources are to make something, and how much (and what kind of) labor goes into it.
You're only describing the supply curve. There's a very limited supply of Obama's faeces and the labour involved is very highly paid, but the price is nevertheless approximately nil. You have to take account of how much people want a thing, not just how much of it there is and how expensive it is to produce.
I see that as an issue too: I expect that after a singularity, money and thus GDP figures would become meaningless.
There are more sources of hard-to-predict weirdness during the transition period. Today, GDP growth partly represents increasing physical quantity of products, but in a large part increasing quality, which fetches higher prices; also, much of the GDP is services. Higher quality sometimes shows up in statistics as higher prices (high-quality food can cost several times as much as cheap food, even with similar ingredients), but not always: computers have become much better over the decades, but that doesn't show up in their prices. If instead of buying some services you buy cheap robots, that may not show up in the GDP statistics as growth.
I'm not sure why polymarket is so off but the Augur based prediction market is still fighting to resolve the Trump bet as well, Trump supporters seem to be throwing money at the dispute function of augur, which is only really able to delay the inevitable and waste their money as the cost to delay ramps up. If I had to guess why people are selling the nTrump tokens at 92 cents on the dollar it's because people want to re-enter the crypto market and may be willing to eat an 8% loss to get out of limbo. I'm really wishing I had sold my nTrump tokens when they were worth 95 cents after the election as I've missed out on nearly $10k in opportunity cost because the funds used to be in ethereum. which was worth less than $400 when I made the bet and is worth $1700 now.
There also may be some fear that the Trump supporters can actually break Augur, although I doubt they can.
The whole of the ethereum ecosystem is going through some very significant pain right now as the gas price is insanely high and the upgrades that will enable scaling are months or years away. It's an unfortunate time to be looking at ethereum based prediction markets, huge fees just for moving coins around let alone interacting with smart contracts that multiply fees makes the whole practice difficult and probably contributes to the current bad price prediction.
I decided that the cost in terms of my time and risk I would make a mistake and lose all my money was too high, especially since it looked like the high returns were only in a few situations and most people would get more stock-market-esque returns.
"There will be an ambiguous resolution if there is no 4 year (nor 1 year) doubling interval by 2050, to isolate specifically the takeoff speed from other things like the chances of no takeoff occurring at all or human extinction." (from the "world output doubling" question)
I love the scrupulous thoroughness of specifying how the bet will resolve in the event of human extinction.
One of the benefits of Metaculus is that since people are predicting for fun, you can generally assume they'll give their true belief in questions that will never resolve due to humanity extinction, Metaculus no longer existing, etc.
"Always bet as though the market will keep existing" is a standard piece of investment advice. Markets suck at pricing in nuclear wars that will make the markets stop trading or make the traded shares worthless.
In a similar way, there's an incentive to predict that AGI (or nuclear war, or super-plague, or asteroid impact, or supervolcano eruption, or whatever other catastrophic or existential threat) is a long way off, because every year with no AGI is a year in which you get to boast about your insightful cyncism, whereas the year you're wrong you have much bigger things to worry about than Fake Internet Points.
A few other people have said this as well, but I think this comment captures it most succinctly, and is definitely important context missing from the original post.
These Fake Internet Points are only worth anything at all as long as metaculus exists when the question resolves, so pretty much any predictions more than, say, 10 years in the future are even more symbolic than the fact they're based on Fake Internet Points would suggest.
Once again I am a̶s̶k̶i̶n̶g̶ ̶y̶o̶u̶ ̶f̶o̶r̶ going to mention my suspicion that the eerily straight lines that economists draw through the last 300 years of economic progress are due to the economists having carefully chosen a lot of tuning parameters so that the lines would be straight, and not at all proof that discontinuities and rapid surges in economic productivity/wellbeing don't happen. I am going to keep mentioning this until somebody replies to my objection in a convincing way. Until then, I am going to assume "GDP over time" charts are a hilarious in-joke amongst economists.
Well, one way to test your hypothesis would be to dig into the papers that present such lines and see for yourself what the formulae are. Let us know what you find.
I'm extremely skeptical of AGI (meaning that I think the currently projected timelines are ridiculously optimistic), but sadly I'd have to bet on the "AI NYT bestseller by 2030" claim. Unfortunately, this problem is being solved from both ends: AI is getting better, and bestsellers are getting worse. It is not inconceivable that, by 2030, just having a semi-coherent book with mostly correct grammar would be sufficient. That's just my subjective opinion, admittedly.
I wish there were a working combinatorial prediction market, real-money or not.
It is of course computationally intractable in theory but I think people can make it work? There seems to be some academic research in this area (seemingly from a few people at Microsoft), but no currently-active public implementation.
This would solve the obvious problem, very apparent on PredictIt, where one combination of events is equivalent to some other combination of events, yet they have different prices.
It would also just be very cool, and probably fun to trade on (although of course less opportunity for arbitrage).
The Trump markets (and related markets such as the John James market) really shook my belief in prediction markets. I defended them when they got the 2016 presidential election wrong because almost everyone else also got it wrong. But I couldn't believe it when people continued to bet on events that had already happened, elections that had been certified well beyond the threshold for recount.
I think there was a perfect storm of many very committed irrational Trump supporters willing to use the prediction markets to express what they wanted. This doesn't shake my faith much. I made a few thousand dollars betting against them. If I'd been allowed to bet more, I would have bet more. So would other people, and eventually the market would have equilibrated.
Again, if there's a prediction market without a maximum allowed bet, one of two things will be true: it's at least as smart as you are, or you can use it to get arbitrarily high amounts of money. Either way sounds pretty good.
So, the tragic fundamental issue with prediction markets is that they require ETH smart contracts. Ethereum gas fees are incredibly expensive. There are many different approaches to fixing the scaling problem here, with plasma and optimistic rollups being some of the more promising ones in development. Ethereum v2, which is apparently going to start rolling out this quarter (!?!), should also greatly improve scaling in a way that should trickle down to smart contract prices. (On the flip side, demand could increase sufficiently that unplatable fees would still be an issue).
Polymarket in particular is completely centralized and has broken prices in a few different ways, with many markets being illiquid or otherwise stuck (though it might use being centralized to lower some fees? not sure). The much more interesting alternatives are Omen and Augur, with Augur being the only completely decentralized resolution market. https://omen.eth.link/ used to work but doesn't come up for me right now; last I checked augur was more or less unusable on its own, but clever people managed to set up a Balancer market to do this with relatively lower fees at catnip.exchange.
I expect Augur to become more impactful in the next year or two as Ethereum gets its shit together.
Polymarket is only partly centralized. Resolution is centralized, trading is decentralized. They can't take your funds but they can resolve questions wrong.
I've been using Polymarket since summer and have earned a significant amount of money on the election's market.
Regarding 6% chance of Trump's comeback, there's a general reason why prediction markets can't be very precise. This 6% market translates into actual 3-4% of profit, and for a market that runs for any length of time this might be worse than just investing your money in S&P. This is not the case for this particular market though, since it will be resolved fairly soon and 3-4% profit in a month is quite a good deal.
There's also the inherent risk, as with any crypto stuff, that via either malice or incompetence some part of the system will break down meaning you don't get your money back.
I’m curious about something regarding the idea of taking prediction markets seriously, hoping someone more well-versed in this will indulge me—
Doesn’t the theory of prediction markets kind of rest on the notion that people will be rational actors placing their bets in such a way that it’s their sincere best guess at whether the prediction will come true, so they can make the maximum amount of money - and doesn’t this make them very vulnerable the moment they have consequences beyond who wins the bet? Suppose a government decides to implement COVID policy based on a prediction market - won’t that immediately just cause people to flock to the markets to “predict” that whatever policy they want enacted will save more lives? At that point, isn’t it just kind of a roundabout way of voting on the policy, except now only people with money to burn get a significant vote?
TL;DR: market price does not map that easily to probability for some contracts, including all AGI/GDP related questions. If we have a simple binary contract paying 0 or 1 on a given date depending on whether a certain event occurs, price of this contract in a liquid market may not reflect market perception of event probability for several reasons. The main reason is that if the outcome is negatively correlated to the value of money, market price will underestimate the probability and vice versa. For example, if we had a contract on the event "Earth is wiped out by hostile aliens before 31/12/2021", a rational actor with unlimited free collateral would be ready to sell this contract at any price irrespective of perceived probability, as there would be no state of the world in which they would have to pay out anything. A more subtle example closer to the AGI questions would be a contract on "Inflation in USD is above 100% per annum in 2021". If the contract settles at 1, the value of a dollar won by a long contract holder is less than 50c in today money; if the contract settles at 0, the value of a dollar lost by a long contract holder is likely to be much higher. Thus, a rational market participant would bid for this contract far less than the perceived probability. Now, today value of $1 in a year of 100% GDP growth caused by AGI is pretty uncertain as nominal prices will be all over the place in such a year. Inflation for some consumption baskets may be -99% and for other consumptions baskets it may be +5000%. So, at best, contract price reflects the joint distribution of likelihood of the contract event and marginal market participant predicted inflation for their consumption baskets. At worst, the price does not reflect anything as people as not very good in understanding this convexity.
Am I reading it right on the 'Will AI surprise us' question that it's predicting a 75% chance it will? This is based on whether other prediction market bets will move? If the market is 75% sure that those other bets are off, why haven't they corrected themselves to reflect that? Is this just an artifact of the low volume/market inefficiency nature of this and other prediction markets that could be corrected if they were made generally legal and widespread?
What does "corrected themselves to reflect that" mean? It seems possible to believe "if AGI comes, it'll appear very suddenly, AND it has a 70% chance of coming". So you'd want the "will AGI market come" to be at 70%, but you recognize that IF that market ends up at 100, it just moves quickly toward that (but you still only think it has a 70% chance of going to 100).
Basically, any market should be at some point between 0% and 100% right now, but as the outcome becomes clear it should tend toward one of those endpoints. The "will it surprise us" market is just a statement about how quickly it does that movement, but the movement to an extreme itself is part of every market (assuming the markets aren't marked "resolved" first).
Possible illustrative example: You could have two markets, "will California have a magnitude 6 Earthquake this year" and "will that Earthquake surprise us?". The answer to the first is probably something like 20% (or something, not important), but the answer to the latter is "yes". We would absolutely expect the prediction market for "will the Earthquake happen" to jump suddenly to 100% once the Earthquake happens. That doesn't mean the "will it happen" market should be higher though, it just means the event will happen suddenly.
GDP growth is far more likely to be limited by the rate of change of human behaviors (slow) and human values (even slower) than by technology or knowledge. You'll probably have to wait for a few generations to die out before you can possibly see any kind of real transformational change.
For example, the growth of lab-grown meat is going to be heavily limited by acceptance, not price or quality. There is nevertheless a stable equilibrium where lab-grown meat is all anybody has ever known and at that point, the prospect of taking a cow, growing in a way that is hugely destructive to the environment before slaughtering and dismembering it, only to obtain something that is indistinguishable from what you already have but far more expensive, will seem entirely preposterous. It will take several generations to get there, however. Superintelligence will not change this.
Similarly, approximately half of all wealth in the US is housing wealth. Think about this for a second! Half of all the stored value in our society relates to where and how we live. It is not the lack of a superintelligence that is preventing this from growing. Growing the amount of housing is going to hit hard caps in terms of land availability. Growing the value of housing at constant size is going to hit caps on the amount of value a person can get from a small living space. Growing the non-housing share of GDP is going to involve getting people to care about things other than their small, crappy house, but this is going to be a huge and slow shift in human values. There may be higher value equilibrium states that we could reach, but getting to them will involve reorganizing extremely large numbers of people in terms of where and how they live. This will not happen quickly, no matter how many supercomputers you have.
Oh wow. This Cyberpunk novel on the Onion Futures Ban writes itself. According to wiki, In 2010 the motion picture association of America successfully had box office receipts added to the futures ban. Apparently some financial firm called Cantor Fitzgerald planned to start a real money Box Office receipts futures market in the 1990s, but the firm's corporate office was destroyed on 9/11. They were just two floors up from where one of the planes hit. Later this guy named "Robert Swagger" tried to start Media Derivatives Inc in 2007, but then the financial crisis hit and the MPAA got box office futures banned as part of the Dodd-Frank Act. What are they trying to hide?
It's not just onions that it's illegal to have futures contracts on; it's movie tickets too.
Robin Hanson has talked about this before:
https://www.overcomingbias.com/2010/03/masking-movie-manipulation.html
https://www.overcomingbias.com/2010/04/democracy-in-action.html
https://www.overcomingbias.com/2012/11/zitzewitz-the-wise.html
Or, here's a more recent, longer article about it: https://www.theringer.com/movies/2018/11/15/18091620/box-office-futures-dodd-frank-mpaa-recession
So, Kalshi also won't be opening markets on what movies will do well, I guess.
(Yes, this is stupid.)
I actually think we're really lucky that speculation in onion futures was banned like that. Without it we' only see the financial disturbances that speculators cause and not the ones they fix. With onions, though, we have this nice cautionary tale that stays relevant and is something important enough to be a test of the idea but not so important as to cause real problems.
What's the cautionary tale that shows the disturbances they fix?
If you're a carrot farmer, you can sell carrot futures six months in advance to the local carrot cake factory and know how much you'll get for them. Both the farmer and factory know what price they'll sell/buy the carrots for and can get on with farming/baking.
If you're an onion farmer, you have to grow the onions not knowing what price you'll get for them when they're harvested. You're exposed to the risk that onion prices will fall and for the owner of a bhaji factory, there's the risk that the price will rise suddenly.
Onion prices are incredibly volatile compared to other agricultural commodities.
And apparently the Onion Futures Act was the direct inspiration for trading in pork bellies and orange juice on the Chicago Mercantile Exchange – the onions were a big part of their business!
On the AI GDP questions, you seem to have left/right swapped.
This paragraph
> The one on the left is pretty weird. Right now world GDP is 80 trillion dollars.
Thanks, fixed.
Huh, I've heard of "onion futures" being illegal before and didn't realise it was literally onions. I always assumed it was a type of future named metaphorically after onions - like futures on futures on futures on something - something with layers. TIL.
Also, as mentioned in my other comment, movie ticket futures are also banned: https://www.theringer.com/movies/2018/11/15/18091620/box-office-futures-dodd-frank-mpaa-recession
Yes, onions and movie tickets are singled out specifically...
Another prediction market site that right now only appears to have two things, but I vaguely remember being pretty liquid for the Trump election: https://ftx.com/markets/prediction (I have never used it)
What do you think Metaculus would predict about its own demise? E.g. something like "predict the year in which Metaculus's monthly active users drops below 100 for 6 consecutive months." And how do you think this prediction effects people's other predictions? Do people care less about being right if they'll be right only after the site is no longer operating?
Here's my answer https://www.metaculus.com/questions/3399/when-will-the-last-metaculus-question-resolution-occur/
https://www.metaculus.com/questions/841/will-metaculus-exist-in-2030/
Thanks
For that question they include this note:
"Due to the nature of this question the best point optimizing prediction would be 99%, no matter of the real probability of Metaculus existing in 2030, as a non-existent Metaculus cannot make you lose points. Still, players are urged to predict in good faith."
Weakened versions of that note also apply to lots of other long-term questions too and will certainly play some role in how people predict.
On the AI-written book question - would a volume of short stories count? I would bet that GPT-4 could write a reasonably coherent short story, especially if you prune the more outre outputs .
>especially if you prune the more outre outputs .
Yeah, this is the important caveat.
Monkies at typewriters will eventually write a bestseller, *if* you have a competent human author/editor choosing which of their texts to try to publish, or which individual sentences/paragraphs to string together and in what order.
'AI will produce trillions of text strings and humans will find one that can be sold' is very different from 'AI will be told to write a best-seller and succeed on first try.'
Human-written books that make it to publication are also selected from among lot of low-quality writing.
I think the most straightforward explanation for why the Polymarket Trump contract is still trading at 6% is that 1) people who aren't familiar with crypto are put off by the complexity and inconvenience of getting their money into USDC so that they can trade there, and 2) people who are familiar with crypto have no interest in locking up their capital for two months for a 6% return, because there are much better opportunities for yield currently available in that space.
Is this true? 6% per two months is 40% per year. The amount of crypto knowledge it takes to invest in Polymarket is minimal - I think I could do it myself if the relayer was working. I don't think it's generally agreed that a rando with a tiny amount of crypto knowledge can make 40%/year risk-free-ish returns? (I think just buying and holding Bitcoin might plausibly be 40%/year, but not risk free).
How much are the fixed costs on it? At the amount of money you're betting, what is the actual expected return assuming there is in fact a negligible chance that Trump will be President on Mar 31?
That wasn't an argument, it was a question. Also, I have no idea what you're getting at.
I believe it costs 2% to enter and 2% to exit a position, so a 6% discrepancy is only exploitable for a 2% profit. These fees are paid to liquidity providers, not the platform.
No, I don't think someone with minimal crypto knowledge could necessarily do significantly better than 6% in two months. But I think the set of people who are familiar enough with crypto to invest in Polymarket, but not familiar enough to have any ideas for other low-risk strategies in crypto, is pretty small.
A really simple alternate strategy would be to hold Bitcoin and short the March futures contracts on a crypto exchange. Right now those contracts are trading roughly 4.2% above the current BTC price (46266 vs 44412, see https://ftx.com/trade/BTC-0326). So you'd have a "risk-free" return of 4.2% in roughly the same time that the Polymarket contract nets you 6%. Also, note that Polymarket is quoting 6%, but if you actually type a number in the box to buy, the "max return on investment" that it shows you is ~4.7% (and that number goes down as the size of your bet increases). So you can do basically as well as the Polymarket bet with an extremely simple futures strategy.
Can you explain what's going on here? Why is the Bitcoin futures contract consistently so much more expensive than Bitcoin?
It's because people are paying for leverage. If you buy spot Bitcoin, you can only take a position that is as large as the amount of money you have in your account. On the other hand, if you buy futures contracts, you can take a position several times larger than your total balance. But because lots of people want to do this, they end up bidding the price of the futures above the value of Bitcoin. It's comparable to borrowing money to buy more spot Bitcoin and paying interest on the borrowed amount (and actually, there is a direct relationship between the futures contango and margin interest rates).
So is the apparently free money I would get by having+shorting Bitcoin to compensate for the risk I'm taking on that these people won't actually be able to pay me later?
Not exactly, because collateralization is handled by the exchange rather than the individuals who are "providing" leverage by shorting futures. The exchange will automatically liquidate traders' positions if their account balance falls too low, and will backstop losses that do occur if liquidations happen too slowly. Here is how FTX does it: https://help.ftx.com/hc/en-us/articles/360027668712-Liquidations.
So you are taking on risk, but the risk is not so much that the individual leveraged traders will go bankrupt. It's more like the risk that the exchange fails to manage its liquidation engine appropriately and also does not have enough capital in its insurance fund to cover a sudden cascade of late liquidations. There have been a few times when this has happened - the last instance of socialized losses that I remember was on Okex in 2018, see https://news.bitcoin.com/okex-socializes-loss-from-over-400-million-bet-among-btc-futures-traders/. But crypto exchanges have gotten pretty sophisticated over the past few years, and so I think this type of risk is minimal and not the main reason for the existence of the "free money".
Can you recommend a good place to short crypto futures? It seems that FTX is unavailable in the US.
Unfortunately, all the exchanges that offer crypto futures are unavailable in the US. They do this to avoid running afoul of US regulators (for a very colorful account of what they're trying to avoid, see https://www.vanityfair.com/news/2021/02/the-rise-and-fall-of-bitcoin-billionaire-arthur-hayes).
That said, it's pretty common for US-based individuals to use those exchanges anyway - you just have to do so with a VPN so that your IP address is located elsewhere. FTX will let you sign up with just an email. The main downside is that if you don't provide photo ID when signing up, you have a maximum daily withdrawal limit of $9000. This may or may not be a problem, depending on how much you're interesting in trading and how patient you're willing to be when you decide to eventually withdraw your money.
Interesting; thanks for the information. I'm comfortable with the VPN and the withdrawal limit, but I'd be concerned about the risk of getting my account shut down. Do you have a sense of whether they can just cut off your access to the account if they find out that you're based in the US?
I'm hesitant to give you concrete recommendations here because I don't want to encourage you to do something that ends up losing you a lot of money. But I do know several people whose accounts have been flagged as belonging to US citizens. In each case the exchange's response was to require them to immediately liquidate their account and withdraw their funds, as opposed to confiscating those funds. I think the exchanges view this as a much better way of limiting their regulatory liability than holding on to users' assets. That said, this is merely anecdotal and I'm not sure that most exchanges have an official policy one way or the other.
You might consider creating a "burner" account with a minimal amount of capital on whatever exchange looks most interesting to you and then test what happens when you out yourself as a US citizen.
I think the CME has offered bitcoin futures for some time and in that last week or so has started offering ethereum futures.
It's more like a 4% return per two months after you factor in the various fees, which would be about 27% a year. Second, it's not risk-free. I don't have any specific concerns about Polymarket (I've been betting through them for a few weeks now), but any kind of internet betting site has some built-in risks that don't exist with more traditional investments.
However, the most important point is that it's not something that you can easily repeat six times over the year to actually earn that 27% return. If I thought there would be similarly safe bets available all the time, I'd absolutely go to a lot of effort to capitalize on them. For a one time bet that will only net me 4% profit and expose my money to some level of risk, it's not such an easy choice.
Am I right that you're hypothesizing that right now Polymarket is unusually exploitable, and you're expecting most times not to have so clearly wrong a contract? If so, what makes you think this?
Polymarket just launched, so it's less liquid, and thus less of an efficient market.
Most of the "safe bet, decent return" markets are based on politics and basically exploit the irrationality of Trump supporters. I imagine this money will slowly drift away as Trump gets further and further from public life.
I actually think it's really just that single market that's unusually exploitable. Even the very similar "Will Biden be president on 3/1" market, will only get you about a 1.5% return, and that's a bet you actually have a chance of losing, so you shouldn't stake your entire bankroll on each bet. If you factor in your change of losing (let's say 0.3 of a percent chance that Biden is not president by 3/1), and you bet using the Kelly criterion, you're down to an expected return of 15% a year. That's still pretty good, but it's not outside the realm of investments that are available elsewhere if you're willing to accept some risk.
Also, I have to correct my earlier statement about the fees. Some of the fees are a one-time cost of getting the money into Polymarket, so they're largely irrelevant if you plan to bet repeatedly over the long term.
I think it would be a mistake to use annualized returns for prediction market forecasts. It is not likely that you can earn 6% on a bet this certain in a prediction market into perpetuity. The pricing on "sure things" over election night on PredictIt was about 5-7% (i.e. CA to vote Democrat), many of them settled within a week, does that mean the expected return on predictit was 1200% annualized? I mean, in one sense literally yes, but not in any useful sense. The return was a one-time return of 5-7%, enough to cover the exchange's fees, and some post-election horizon expected return that would be positive but vastly lower.
Likewise now, yes you can earn 6% in the next 2 months on Polymarket. I don't know what the risk associated with the platform/stablecoin is exactly, but its clearly the main risk you are taking. After those 2 months, will there be another 6% return in two months on something just as certain as this Trump market? Unlikely. So you have to do work on this crazy crypto market to get comfortable with it, put up enough money upfront to make that work worth your time, and take this concentrated platform risk all to earn <6% expected return as a one-time event, and considerably less on an expected basis over time.
When you see a $20 bill on the sidewalk, do you ignore it because you don't expect there to be another $20 bill on that sidewalk next month?
In this case, you know there won't be - not for another two years at least - since it was predicated on the election. On top of that, it's not really a $20 bill on the sidewalk, but a $20 bill at the other end of a mudpit, but you have to put $300 into a box for a couple months, and there's a toll of 2/3 of the $20 bill to be allowed to cross the mudpit.
Well, it stretches the analogy a bit, but it's more accurate. You have to tie up your money to get the return, doing it is somewhat confusing, and - worst of all - the market (according to other comments) has a 2% deposit fee and a 2% withdrawal fee. Oh, and it's an online cryptocurrency site, so there's a nonzero chance that they'll just disappear with all of your money.
The issue is that the effort I spend learning how to work this spot-bet, even if it's a sure thing, is wasted effort if I can't repeat it.
If I think I can do arbitrage between crypto networks several times a month, I can justify investing time to learn about this latest nerd thing.
To push that market down, given that there are still crazy people putting money into yes, you need to risk a lot of money on a platform that hasn't been tested enough.
Check this out for example (about Augur but still) https://twitter.com/VitalikButerin/status/1326788320173690880
This is not quite accurate. The best low-risk DeFi yields are maybe in the 12-20% range.
I haven't maxed out the Trump one simply because I'm maxed out on the Biden one that resolves Mar 1st, and I want to keep capital liquid to take advantage of other markets that arise.
This is true if you are only looking at strategies that are very close to being "risk-free" (although see the futures arbitrage strategy I mentioned in my reply to Scott). But if you're willing to take on and manage a moderate amount of price risk, you can do a lot better than 12-20% APY.
Another way to put it is that any decent crypto hedge fund/prop shop would be very disappointed to return only 6% over the next two months, and they have capacity constraints that the average individual investor doesn't have. To Scott's point, these strategies obviously do require some level of sophistication and knowledge of the crypto sector, but my point was that I think most people who would consider trading on Polymarket in size probably clear that bar.
If you run a hedge fund, you capture ~20% of wins and are responsible for ~0% of losses.
You don't want to lose, of course. But your incentive to avoid small risks is a lot less than the incentive of someone putting in a million of their own hard-earned greenbacks.
The futures strategy is interesting but I suspect there's some margin risk with large swings? Have to look into it.
Sure, and I agree that hedge funds as a group are not well-incentivized to avoid tail risks. But I think most prop shops are also aiming higher than 6% by end of March, and for them it's their own capital at risk.
For the futures strategy, there is no margin risk if your short position is equal to your spot position and you hold both positions at the same exchange. The only risk is counterparty risk, i.e. the risk that the exchange you're trading on gets hacked/compromised/shut down. This is a legitimate concern, especially when your upside is just a few percent, but it also applies to trading on Polymarket.
This doesn't explain why people are buying at 6%
The May/June 2020 upward deviation in predictions (Will AI progress surprise us? / Date first AGI is publically known) coincides with the release of GPT-3.
For "Date First AGI is publicly known" and similar:
It seems like this is a sort of problem where the time value of money really matters. I.e., a bet that yields $100 in 2025 is worth more than a bet that yields $100 in 2055 (even after adjusting for inflation), so there is the potential of having a huge thumb on the scale that encourages people to bet on sooner dates rather than later dates (or else encourages the "soon" predicters to bet and the "later" predicters not to).
How do metaculus or real-money sites account for this? Or do they even need to from a business perspective?
On Metaculus you win fake internet points and not money, so that's how they solve that problem. It's obviously both a pro and a con that real money isn't involved, but it works remarkably well considering.
Are you limited in the Metaculus Fake Internet Points you can wager? If not, that's a bit more comforting, but I think I would still be pushed toward betting on sooner outcomes because my discount rate for MFIP is very high.
You don't have to spend points make a prediction, a prediction wins you positive points if you're close to the mark and loses you points if you're wrong, but doesn't cost to play. So a discount rate shouldn't come into it, the only opportunity cost is on your time, not your points.
The downside of this is that "smart money" can't come in and correct a large error merely by staking more - all participants are weighted equally in the "community prediction" (actually they're weighted by how recent their prediction was).
Metaculus isn't technically a prediction "market" given you're not buying and selling these things - you're just winning points for correct predictions.
I'd like to see it gain more market-like properties, but it does work quite well in its current form.
That definitely makes things better than a dumb model of what a true prediction market does (I'm still interested in what they actually do), but I still think that there is an incentive to bet on things that pay out now. I.e., the only way that I'm going to spend any time on Metaculus over the long run is if I have a high point total that I can show off. If I bet on a bunch of stuff to happen in 2050, I'll never be around to enjoy it (maybe I'll be dead, but certainly I'll have quit Metaculus because none of my bets have paid off yet and I don't have any cool MFIP to show off). So regardless of when I think X will happen, I am incentivized to bet on them happening in the next few years.
Do they have payoff structures that mitigate this? Again, as a business model they may not need it, but it seems like something to think about if we want to draw conclusions about reality from Metaculus predictions.
> all participants are weighted equally in the "community prediction" (actually they're weighted by how recent their prediction was).
Wouldn't it make sense to give more weight to predictors with a good track record?
the "community prediction" by definition weighs all participants the same. There is also the "metaculus prediction" which weighs users differently depending on track-record, though the exact algorithm is secret. However, the "Metaculus prediction" isn't visible until a question has closed. Most questions on Metaculus close prior to being resolved, so this is usually a decent chunk of time where you can no longer make predictions, and can see the "Metaculus prediction".
I think a question closing for predictions prior to resolution is designed to incentivise users to make an as-accurate-as-possible "final" prediction when there is still some uncertainty (you can update your prediction any time prior to a question closing). It's not a great mechanism IMHO and I think I'd prefer your reward to just be based on equal weight per unit time you're in each question without the "final" prediction being more heavily weighted. It's annoying that as an event gets closer the questions often close such that you no longer can use metaculus to see what the likely outcome is closer to the event.
I think the "metaculus prediction" is hidden prior to closing to prevent some kind of gaming - maybe they don't want users to simply copy the predictions of the most reputable users, perhaps this can lead to "bubbles" of false consensus, I'm not sure.
But yeah, I'd like to see it become more market-like such that we can a) view a prediction all the way up until an even occurs and b) allow heavier-hitters to move the prediction more.
You might think that this doesn't matter for Metaculus, because it's after all meaningless points, not money, but it's actually the reason I didn't bother with Metaculus after the first time I saw it. I can't be bothered with stupid internet points a decade from now.
Yeah. And since most people are going to think the same way I'd worry the outcome for long term predictions is just the same as if you got a group of people to submit probabilities, with no reward mechanism for correct results
Also, you have to assume that the traditional economy and the specific currency you'll be paid in would meaningfully survive a hard take-off scenario to begin with. Which I personally don't.
So, yeah, lots of room for considerations about what the money will mean in the event of the prediction going one way or another to take into account here.
Disclaimer: non-expert opinion; I got some USDC onto Polymarket last week and have read their docs and lurked their discord, but don't understand every detail of how it works.
Polymarket's transfer issues are recent, and the Trump-related markets have always been unreasonably pro-Trump (in my opinion).
The issue with the relayers is this: due to high trading volumes for ETH, it costs around $50 in gas fees to transfer USDC to Polymarket. Polymarket itself is picking up the tab for these fees, but they don't want to pay more than necessary. Therefore, they're only filling the gas tank when gas prices are relatively low, and they're not putting in enough for everyone who wants to process a transfer to do so.
Unfortunately, there is no queueing system, so your USDC will not be moved from the first relayer through to your Polymarket account until you click the button to do so when there is money in the gas tank.
There is no way to retrieve your USDC from the relayer except moving it through to your Polymarket account and then back out.
You can add money to the gas tank yourself to try to hurry things along, but it will cost at least $50 or so, and you should read up on the discord before doing so.
The Polymarket team is working on switching to a much cheaper transfer system, but it's unclear how long this will take.
Why does Trump to be President trade at 6%? The question has two parts: (1) why is anybody crazy enough to bet yes on that, and (2) why arent people eager enough to offer it at a lower price.
Subquestion 1 is really an anomaly, but one that clearly exists in any prediction market and in the real world.
Subquestion 2 has an easy answer -- Polymarket might not have the value-at-risk cap and trader cap that PredictIt has, but it does have some real risk of being shut down or declared illegal at any moment. PredictIt has a no action letter from the feds, Polymarket does not. Some combination of being a pain to get started with, being crypto-based, not having any assurance from US authorities that the site will not find itself in some state of illegality, and other operational risks mean that nobody is willing to put a large amount of money there without some minimum expected return to compensate for those risks.
2: and technical risks.
As someone currently ranked in the top 50 on Metaculus having joined 6 months ago, and who has made >$100k on Polymarket having joined around 1.5 months ago, I have some thoughts.
First, on "But most of Metaculus is bread-and-butter questions about when such-and-such a system will meet such-and-such a benchmark, or how many parameters it will take, or things like that." Those questions are almost all from a specific contest run by the Open Philanthropy Foundation with $50k in prizes. See https://www.metaculus.com/ai-progress-tournament/
They do not actually represent the bulk of Metaculus questions, but it may appear that way if you sort in certain ways.
Second, Metaculus heavily rewards predicting on lots and lots of questions. If you just copy the community on every question, you gain net points in expectation, and do better if you can beat them, which you can do pretty easily just by looking at older questions where most predictors haven't updated recently. For two examples, see this thread https://twitter.com/avi_eisen/status/1357385286985121792. Often in practice, someone happens to see an older question and the community prediction looks wrong to them, they write a comment, which bumps up the question so a bunch of other people now update. Because of this dynamic, you need to be very careful if you're going to try to use Metaculus numbers as a prior/input into your model/etc. There are often fairly uninformed predictions simply because nobody has done a deep dive on the question for two months. I myself have predicted on 881 questions to date, and it's nearly impossible to update them all regularly. I try every other month or so and my brain just goes numb after scrolling through a few hundred and thinking which might have changed odds.
Third, as you can probably guess from my intro, it is absolutely worth it to figure out the kinks with Polymarket. The market is far from efficient. However, it is harder to beat than Metaculus, unless you're betting on the odds of a Trump coup.
One thing I did was use Metaculus questions to guide my betting, which was interesting. See these three questions:
https://www.metaculus.com/questions/6293/biden-in-person-inauguration/
https://www.metaculus.com/questions/6047/1m-lost-in-prediction-market/
https://www.metaculus.com/questions/5684/states-flipping-in-2020-election/
Very interesting. Out of curiosity, how much capital did you put up to make that ~$100k in Polymarket?
On the order of 500k, which was greater than my liquid net worth. My personal finance is a bit complex right now, but let's just say that it was worth paying what would normally be considered high interest rates, given the risk levels here.
That is absolutely fascinating to me. Can you share more how you got started? Do you do something similar by profession (finance, statistics, data scraping)? With your skills, would you get paid more in a traditional job than if you were given 10k USD and told to go wild?
This is the closest thing I’ve seen to somebody literally been payed to be an oracle.
The bulk of the Poly money was from knowing in January that Joe Biden had won the election and was going to be inaugurated on the 20th, and being willing to take action to convert that knowledge plus cash into more cash.
I don't know if I have a long-term edge after that kind of stuff is factored out. I've never been too interested in a finance job because I felt the competition was too stiff and I prefer industries where I have an edge, which certainly applied to the political Poly markets. I've never had a traditional job.
Scott, if it interests you, the site Papers With code has a nice plot that shows the state of the art on these problems, and explains the datasets. I think you might find at least some of the problems to be exciting.
The SQuAD dataset is a dataset of questions and answers based on wikipedia articles, and is described here: https://paperswithcode.com/dataset/squad
The Mini Imagenet is a subset of Imagenet, a dataset of a thousand classes of images. One shot learning is "learning to recognize X from a single image of X", which is pretty cool and non trivial thing (in my opinnion) for a neural network to do. It is described here:
https://paperswithcode.com/sota/few-shot-image-classification-on-mini-1
Metaculus doesn't seem to expect any big changes in these domain in the next year, which is a bit unfortunate.
"Presumably some AI-related advance short of AGI would cause an unprecedented world economic boom. I am pretty skeptical of this because there have been a lot of weird unprecedented things in economic history and they have never caused that severe a discontinuity."
We've never had one advance that affected every industry almost all at once; they've always had to diffuse out at the rate of human learning, human decision making processes and institutions, manufacturing and/or building infrastructure and logistics networks, and so on. That will still be somewhat true in a less-than-FOOM-takeoff, I think, but it doesn't mean there can't be a huge shock. Part of the value of AI is automating decision making, probably reducing those limitations. Wars and depressions do cause huge negative shocks, often followed by recovering faster than trend. In the Great Depression, US GDP fell by 30%. WWII was followed by a 30 year economic boom.
I will say I'm also quite confused about what I should expect to happen to prices and nominal GDP in a world where almost everything gets automated. My current understanding is that however indirectly, prices today are mostly about how fundamentally scarce the resources are to make something, and how much (and what kind of) labor goes into it. I expect asymptotically AI pushes the value of (non-human) labor to zero, and costs depend only on material composition and energy for production and maintenance, but the intermediate stages could be super weird in a slower takeoff depending on which labor components get zeroed out in what order and what prices of still-scarce-things we choose to bid up in response.
> My current understanding is that however indirectly, prices today are mostly about how fundamentally scarce the resources are to make something, and how much (and what kind of) labor goes into it.
You're only describing the supply curve. There's a very limited supply of Obama's faeces and the labour involved is very highly paid, but the price is nevertheless approximately nil. You have to take account of how much people want a thing, not just how much of it there is and how expensive it is to produce.
This is true, I did ignore demand. Doh.
I see that as an issue too: I expect that after a singularity, money and thus GDP figures would become meaningless.
There are more sources of hard-to-predict weirdness during the transition period. Today, GDP growth partly represents increasing physical quantity of products, but in a large part increasing quality, which fetches higher prices; also, much of the GDP is services. Higher quality sometimes shows up in statistics as higher prices (high-quality food can cost several times as much as cheap food, even with similar ingredients), but not always: computers have become much better over the decades, but that doesn't show up in their prices. If instead of buying some services you buy cheap robots, that may not show up in the GDP statistics as growth.
I'm not sure why polymarket is so off but the Augur based prediction market is still fighting to resolve the Trump bet as well, Trump supporters seem to be throwing money at the dispute function of augur, which is only really able to delay the inevitable and waste their money as the cost to delay ramps up. If I had to guess why people are selling the nTrump tokens at 92 cents on the dollar it's because people want to re-enter the crypto market and may be willing to eat an 8% loss to get out of limbo. I'm really wishing I had sold my nTrump tokens when they were worth 95 cents after the election as I've missed out on nearly $10k in opportunity cost because the funds used to be in ethereum. which was worth less than $400 when I made the bet and is worth $1700 now.
There also may be some fear that the Trump supporters can actually break Augur, although I doubt they can.
The whole of the ethereum ecosystem is going through some very significant pain right now as the gas price is insanely high and the upgrades that will enable scaling are months or years away. It's an unfortunate time to be looking at ethereum based prediction markets, huge fees just for moving coins around let alone interacting with smart contracts that multiply fees makes the whole practice difficult and probably contributes to the current bad price prediction.
Just wondering, did you find a way to do Etherium staking?
I decided that the cost in terms of my time and risk I would make a mistake and lose all my money was too high, especially since it looked like the high returns were only in a few situations and most people would get more stock-market-esque returns.
"There will be an ambiguous resolution if there is no 4 year (nor 1 year) doubling interval by 2050, to isolate specifically the takeoff speed from other things like the chances of no takeoff occurring at all or human extinction." (from the "world output doubling" question)
I love the scrupulous thoroughness of specifying how the bet will resolve in the event of human extinction.
One of the benefits of Metaculus is that since people are predicting for fun, you can generally assume they'll give their true belief in questions that will never resolve due to humanity extinction, Metaculus no longer existing, etc.
"Always bet as though the market will keep existing" is a standard piece of investment advice. Markets suck at pricing in nuclear wars that will make the markets stop trading or make the traded shares worthless.
In a similar way, there's an incentive to predict that AGI (or nuclear war, or super-plague, or asteroid impact, or supervolcano eruption, or whatever other catastrophic or existential threat) is a long way off, because every year with no AGI is a year in which you get to boast about your insightful cyncism, whereas the year you're wrong you have much bigger things to worry about than Fake Internet Points.
A few other people have said this as well, but I think this comment captures it most succinctly, and is definitely important context missing from the original post.
These Fake Internet Points are only worth anything at all as long as metaculus exists when the question resolves, so pretty much any predictions more than, say, 10 years in the future are even more symbolic than the fact they're based on Fake Internet Points would suggest.
Once again I am a̶s̶k̶i̶n̶g̶ ̶y̶o̶u̶ ̶f̶o̶r̶ going to mention my suspicion that the eerily straight lines that economists draw through the last 300 years of economic progress are due to the economists having carefully chosen a lot of tuning parameters so that the lines would be straight, and not at all proof that discontinuities and rapid surges in economic productivity/wellbeing don't happen. I am going to keep mentioning this until somebody replies to my objection in a convincing way. Until then, I am going to assume "GDP over time" charts are a hilarious in-joke amongst economists.
Well, one way to test your hypothesis would be to dig into the papers that present such lines and see for yourself what the formulae are. Let us know what you find.
I'm extremely skeptical of AGI (meaning that I think the currently projected timelines are ridiculously optimistic), but sadly I'd have to bet on the "AI NYT bestseller by 2030" claim. Unfortunately, this problem is being solved from both ends: AI is getting better, and bestsellers are getting worse. It is not inconceivable that, by 2030, just having a semi-coherent book with mostly correct grammar would be sufficient. That's just my subjective opinion, admittedly.
I wish there were a working combinatorial prediction market, real-money or not.
It is of course computationally intractable in theory but I think people can make it work? There seems to be some academic research in this area (seemingly from a few people at Microsoft), but no currently-active public implementation.
https://www.microsoft.com/en-us/research/publication/a-combinatorial-prediction-market-for-the-u-s-elections/
This would solve the obvious problem, very apparent on PredictIt, where one combination of events is equivalent to some other combination of events, yet they have different prices.
It would also just be very cool, and probably fun to trade on (although of course less opportunity for arbitrage).
The Trump markets (and related markets such as the John James market) really shook my belief in prediction markets. I defended them when they got the 2016 presidential election wrong because almost everyone else also got it wrong. But I couldn't believe it when people continued to bet on events that had already happened, elections that had been certified well beyond the threshold for recount.
I think there was a perfect storm of many very committed irrational Trump supporters willing to use the prediction markets to express what they wanted. This doesn't shake my faith much. I made a few thousand dollars betting against them. If I'd been allowed to bet more, I would have bet more. So would other people, and eventually the market would have equilibrated.
Again, if there's a prediction market without a maximum allowed bet, one of two things will be true: it's at least as smart as you are, or you can use it to get arbitrarily high amounts of money. Either way sounds pretty good.
So, the tragic fundamental issue with prediction markets is that they require ETH smart contracts. Ethereum gas fees are incredibly expensive. There are many different approaches to fixing the scaling problem here, with plasma and optimistic rollups being some of the more promising ones in development. Ethereum v2, which is apparently going to start rolling out this quarter (!?!), should also greatly improve scaling in a way that should trickle down to smart contract prices. (On the flip side, demand could increase sufficiently that unplatable fees would still be an issue).
Polymarket in particular is completely centralized and has broken prices in a few different ways, with many markets being illiquid or otherwise stuck (though it might use being centralized to lower some fees? not sure). The much more interesting alternatives are Omen and Augur, with Augur being the only completely decentralized resolution market. https://omen.eth.link/ used to work but doesn't come up for me right now; last I checked augur was more or less unusable on its own, but clever people managed to set up a Balancer market to do this with relatively lower fees at catnip.exchange.
I expect Augur to become more impactful in the next year or two as Ethereum gets its shit together.
Polymarket is only partly centralized. Resolution is centralized, trading is decentralized. They can't take your funds but they can resolve questions wrong.
So no one has the ability to update the smart contract?
I haven't verified the code myself, but that's my understanding.
I've been using Polymarket since summer and have earned a significant amount of money on the election's market.
Regarding 6% chance of Trump's comeback, there's a general reason why prediction markets can't be very precise. This 6% market translates into actual 3-4% of profit, and for a market that runs for any length of time this might be worse than just investing your money in S&P. This is not the case for this particular market though, since it will be resolved fairly soon and 3-4% profit in a month is quite a good deal.
There's also the inherent risk, as with any crypto stuff, that via either malice or incompetence some part of the system will break down meaning you don't get your money back.
Yes, of course. I wouldn't make any long-term bets in a prediction market that exists less than a year.
I’m curious about something regarding the idea of taking prediction markets seriously, hoping someone more well-versed in this will indulge me—
Doesn’t the theory of prediction markets kind of rest on the notion that people will be rational actors placing their bets in such a way that it’s their sincere best guess at whether the prediction will come true, so they can make the maximum amount of money - and doesn’t this make them very vulnerable the moment they have consequences beyond who wins the bet? Suppose a government decides to implement COVID policy based on a prediction market - won’t that immediately just cause people to flock to the markets to “predict” that whatever policy they want enacted will save more lives? At that point, isn’t it just kind of a roundabout way of voting on the policy, except now only people with money to burn get a significant vote?
TL;DR: market price does not map that easily to probability for some contracts, including all AGI/GDP related questions. If we have a simple binary contract paying 0 or 1 on a given date depending on whether a certain event occurs, price of this contract in a liquid market may not reflect market perception of event probability for several reasons. The main reason is that if the outcome is negatively correlated to the value of money, market price will underestimate the probability and vice versa. For example, if we had a contract on the event "Earth is wiped out by hostile aliens before 31/12/2021", a rational actor with unlimited free collateral would be ready to sell this contract at any price irrespective of perceived probability, as there would be no state of the world in which they would have to pay out anything. A more subtle example closer to the AGI questions would be a contract on "Inflation in USD is above 100% per annum in 2021". If the contract settles at 1, the value of a dollar won by a long contract holder is less than 50c in today money; if the contract settles at 0, the value of a dollar lost by a long contract holder is likely to be much higher. Thus, a rational market participant would bid for this contract far less than the perceived probability. Now, today value of $1 in a year of 100% GDP growth caused by AGI is pretty uncertain as nominal prices will be all over the place in such a year. Inflation for some consumption baskets may be -99% and for other consumptions baskets it may be +5000%. So, at best, contract price reflects the joint distribution of likelihood of the contract event and marginal market participant predicted inflation for their consumption baskets. At worst, the price does not reflect anything as people as not very good in understanding this convexity.
Am I reading it right on the 'Will AI surprise us' question that it's predicting a 75% chance it will? This is based on whether other prediction market bets will move? If the market is 75% sure that those other bets are off, why haven't they corrected themselves to reflect that? Is this just an artifact of the low volume/market inefficiency nature of this and other prediction markets that could be corrected if they were made generally legal and widespread?
What does "corrected themselves to reflect that" mean? It seems possible to believe "if AGI comes, it'll appear very suddenly, AND it has a 70% chance of coming". So you'd want the "will AGI market come" to be at 70%, but you recognize that IF that market ends up at 100, it just moves quickly toward that (but you still only think it has a 70% chance of going to 100).
Basically, any market should be at some point between 0% and 100% right now, but as the outcome becomes clear it should tend toward one of those endpoints. The "will it surprise us" market is just a statement about how quickly it does that movement, but the movement to an extreme itself is part of every market (assuming the markets aren't marked "resolved" first).
Possible illustrative example: You could have two markets, "will California have a magnitude 6 Earthquake this year" and "will that Earthquake surprise us?". The answer to the first is probably something like 20% (or something, not important), but the answer to the latter is "yes". We would absolutely expect the prediction market for "will the Earthquake happen" to jump suddenly to 100% once the Earthquake happens. That doesn't mean the "will it happen" market should be higher though, it just means the event will happen suddenly.
There's also the Foresight Exchange , which is still fake internet points but at least has been around a bit longer. E.g. http://www.ideosphere.com/fx-bin/Claim?claim=GoCh
GDP growth is far more likely to be limited by the rate of change of human behaviors (slow) and human values (even slower) than by technology or knowledge. You'll probably have to wait for a few generations to die out before you can possibly see any kind of real transformational change.
For example, the growth of lab-grown meat is going to be heavily limited by acceptance, not price or quality. There is nevertheless a stable equilibrium where lab-grown meat is all anybody has ever known and at that point, the prospect of taking a cow, growing in a way that is hugely destructive to the environment before slaughtering and dismembering it, only to obtain something that is indistinguishable from what you already have but far more expensive, will seem entirely preposterous. It will take several generations to get there, however. Superintelligence will not change this.
Similarly, approximately half of all wealth in the US is housing wealth. Think about this for a second! Half of all the stored value in our society relates to where and how we live. It is not the lack of a superintelligence that is preventing this from growing. Growing the amount of housing is going to hit hard caps in terms of land availability. Growing the value of housing at constant size is going to hit caps on the amount of value a person can get from a small living space. Growing the non-housing share of GDP is going to involve getting people to care about things other than their small, crappy house, but this is going to be a huge and slow shift in human values. There may be higher value equilibrium states that we could reach, but getting to them will involve reorganizing extremely large numbers of people in terms of where and how they live. This will not happen quickly, no matter how many supercomputers you have.
Oh wow. This Cyberpunk novel on the Onion Futures Ban writes itself. According to wiki, In 2010 the motion picture association of America successfully had box office receipts added to the futures ban. Apparently some financial firm called Cantor Fitzgerald planned to start a real money Box Office receipts futures market in the 1990s, but the firm's corporate office was destroyed on 9/11. They were just two floors up from where one of the planes hit. Later this guy named "Robert Swagger" tried to start Media Derivatives Inc in 2007, but then the financial crisis hit and the MPAA got box office futures banned as part of the Dodd-Frank Act. What are they trying to hide?
https://nymag.com/movies/theindustry/67275/