When I click on the links to download the data I get a tab that pops open briefly but no option to actually download the data (Chrome, Windows 10). Checking out the URLs I managed to get the actual data files.
I don't care for myself, but just wanted to note that it is very easy to personally identify people from, say, a meetup group located anywhere that is not CA (and many of the survey questions are very personal)
"There’s no such thing as cheating, short of time travel or murdering competitors" -- so murdering people other than competitors, such as subjects of questions, is not cheating either?
You are, of course, aware, that in a community like this, more people will be trying to hack your process than will be trying to generate good answers.
People will be asking questions like: given the information that I am reasonably confident of, how do I create a small enough set of potential response combinations that I can submit them all, that maximizes the expected value of the best performing response?
This is not necessarily about making good predictions.
I should note that I ignored the perverse incentives when playing in Blind Mode. Not just the "if you want to come first in a prediction contest, rather than to maximise your expected score, some degree of overconfidence is optimal" thing, but also the "predict against things which are likely to stop Scott paying you" thing (there are a couple of questions which are unlikely to resolve true without resulting in Global Thermonuclear War).
I entered the Prediction Contest mostly to calibrate how wrong my gut answers were, rather than to try and get them right. So Full Mode loses most of my interest (plus it sounds like...work, potential cash and internet points notwithstanding).
The variance is still fascinating though. Some topics frequently cleave the community at its joints, so I expect that on things like AI risk. Others, it makes me wonder at what kinda priors are running underneath the hood. "What do they know that I don't?" "Why so (un)confident?" (Iirc there was only a single question I felt truly deserved a 1-or-99 estimate., and only a couple true-50s either.)
"Will prediction markets say Ron DeSantis is the most likely Republican nominee for President in 2024?"
Will *prediction markets* say it? Pretty sure they will.
Will he *be* the nominee? Different question, and it depends if he feels that this is a good time to go for it, or would it be better to wait it out, build up his qualifications for campaign, and go in 2028. I think he might wait it out.
I don't think that the people who bet on prediction markets are the same set of people who nominate Republican candidates, and I think it's a case of "what is the current biggest GOP name we hear bandied about, especially as a boogeyman for liberals?" re: will the prediction markets name him.
I downloaded the data and ran some correlations. What I found most surprising was how few things were correlated. The political things were correlated with each other, but most things were pretty uncorrelated. I am interested in seeing what there is with the more full dataset. Anyway, here's some funny correlations I found: (they are all pretty small so don't read into them too much)
What is the best way to find out what my own predictions actually were? Or, equivalently, how can I figure out which line on the spreadsheet represents me?
If I have time I'd like to make a data notebook of this (Quarto) and put it on Kaggle, also maybe run the R Aggrecat package https://osf.io/preprints/metaarxiv/74tfv and share it --
1. Does that make sense, or is that against the spirit of the competition
I jotted down my answers in a file as I submitted them. Now I see they're not in the spreadsheet of submissions. I assume this is user (me) error, but noting it here just in case it's a symptom of a larger issue.
This means I've missed out on my $0.15 expected gain from my shot at the prize money. Or maybe not. Anyone who's participating in the "Full Mode" predictions and wants the added substantial advantage of knowing my predictions can pay me $0.15. Just reply to this comment and we'll make arrangements. I only need one response to break even! I even accept contingency payments...you pay only if you win!
In the interest of truth in advertising, the correlation between my answers and the mean of all answers is 0.79. I wonder what the correlation would be for an optimally valuable additional set of answers? 1.00 corresponds to useless, so I've probably overvalued my answers at $0.15.
When I click on the links to download the data I get a tab that pops open briefly but no option to actually download the data (Chrome, Windows 10). Checking out the URLs I managed to get the actual data files.
XLSX: http://slatestarcodex.com/Stuff/2023blindmode_predictions.xlsx
CSV: http://slatestarcodex.com/Stuff/2023blindmode_predictions.csv
EDIT:
Weird, even those links still open a tab for me that just closes. I can download the data if I:
1. Open the link in a new window
2. Click on the URL in the browser bar at the top of the screen
3. Press 'ENTER'
Weird Substack behaviour?
I don't care for myself, but just wanted to note that it is very easy to personally identify people from, say, a meetup group located anywhere that is not CA (and many of the survey questions are very personal)
"There’s no such thing as cheating, short of time travel or murdering competitors" -- so murdering people other than competitors, such as subjects of questions, is not cheating either?
You are, of course, aware, that in a community like this, more people will be trying to hack your process than will be trying to generate good answers.
People will be asking questions like: given the information that I am reasonably confident of, how do I create a small enough set of potential response combinations that I can submit them all, that maximizes the expected value of the best performing response?
This is not necessarily about making good predictions.
I should note that I ignored the perverse incentives when playing in Blind Mode. Not just the "if you want to come first in a prediction contest, rather than to maximise your expected score, some degree of overconfidence is optimal" thing, but also the "predict against things which are likely to stop Scott paying you" thing (there are a couple of questions which are unlikely to resolve true without resulting in Global Thermonuclear War).
If you want to take a brief look at the predictions, I put together a Google sheet with some simple summaries. One tab has some percentiles for all questions, the other has a selector to view the histogram for any individual question. https://docs.google.com/spreadsheets/d/1jiRhSp7MmPsJp64oMlBHiHqtlm7Ix_u2tWi0XTX_Y7w/edit?usp=sharing
Pretty wild how many questions have a significant number of people at both 99 and 1.
I entered the Prediction Contest mostly to calibrate how wrong my gut answers were, rather than to try and get them right. So Full Mode loses most of my interest (plus it sounds like...work, potential cash and internet points notwithstanding).
The variance is still fascinating though. Some topics frequently cleave the community at its joints, so I expect that on things like AI risk. Others, it makes me wonder at what kinda priors are running underneath the hood. "What do they know that I don't?" "Why so (un)confident?" (Iirc there was only a single question I felt truly deserved a 1-or-99 estimate., and only a couple true-50s either.)
The beginning of the survey says "Five Quick Demographic Questions" but only three demographic questions follow
The CSV link points also to the XLSX data
Hmmm, thing is, for this one:
"Will prediction markets say Ron DeSantis is the most likely Republican nominee for President in 2024?"
Will *prediction markets* say it? Pretty sure they will.
Will he *be* the nominee? Different question, and it depends if he feels that this is a good time to go for it, or would it be better to wait it out, build up his qualifications for campaign, and go in 2028. I think he might wait it out.
I don't think that the people who bet on prediction markets are the same set of people who nominate Republican candidates, and I think it's a case of "what is the current biggest GOP name we hear bandied about, especially as a boogeyman for liberals?" re: will the prediction markets name him.
I downloaded the data and ran some correlations. What I found most surprising was how few things were correlated. The political things were correlated with each other, but most things were pretty uncorrelated. I am interested in seeing what there is with the more full dataset. Anyway, here's some funny correlations I found: (they are all pretty small so don't read into them too much)
Educated behavior:
1. Getting married
2. Flying on planes
3. Having children
4. Driving cars
5. Being high IQ
Uneducated behavior:
1. Believing humanity will go extinct due to AI
2. Considering suicide
3. Supporting Donald Trump
4. Enjoying ACX meetups
5. Being a STEM person (as opposed to humanities)
Vaccinated behavior:
1. Trusting the mainstream media
2. Supporting abortion
3. Supporting feminism
4. Being an effective altruist
5. Flying on planes
Unvaccinated behavior:
1. Supporting Donald Trump
2. Being a Republican
3. Having worse long COVID
4. Being religious
5. Living in the suburbs
Risky behavior:
1. Flying on planes
2. Taking LSD
3. Believing nuclear weapons will be used in 2023
4. Being in a relationship
5. Getting COVID
Safe behavior:
1. Having anxiety
2. Trusting the mainstream media
3. Living in the suburbs
4. Ignoring emails
5. Enjoying bread and pasta
Is there any way that I can review my blind mode answers ?
What is the best way to find out what my own predictions actually were? Or, equivalently, how can I figure out which line on the spreadsheet represents me?
If I have time I'd like to make a data notebook of this (Quarto) and put it on Kaggle, also maybe run the R Aggrecat package https://osf.io/preprints/metaarxiv/74tfv and share it --
1. Does that make sense, or is that against the spirit of the competition
2. Has anyone done this/is anyone doing this
3. Anyone want to collab on this?
I jotted down my answers in a file as I submitted them. Now I see they're not in the spreadsheet of submissions. I assume this is user (me) error, but noting it here just in case it's a symptom of a larger issue.
This means I've missed out on my $0.15 expected gain from my shot at the prize money. Or maybe not. Anyone who's participating in the "Full Mode" predictions and wants the added substantial advantage of knowing my predictions can pay me $0.15. Just reply to this comment and we'll make arrangements. I only need one response to break even! I even accept contingency payments...you pay only if you win!
In the interest of truth in advertising, the correlation between my answers and the mean of all answers is 0.79. I wonder what the correlation would be for an optimally valuable additional set of answers? 1.00 corresponds to useless, so I've probably overvalued my answers at $0.15.
For next time, it would be helpful to get the blind mode answers in a Google Sheet as well, just for those who might want to get a quick sense of it.
I can't see my entry. Nobody with my age and religion. I'm sure I submitted correctly.