Open Thread 167
This is the weekly visible open thread. Odd-numbered open threads will be no-politics, even-numbered threads will be politics-allowed. This one is odd-numbered, so be careful. Otherwise, post about anything else you want. Also:
1: Thanks to everyone who commented about multiple hypothesis testing. I think the answers to the questions I had were something like 1) the vitamin D study should have used stratified random sampling. 2) NHST is not great at combining experiments, but you could potentially do it through binomial probability or the Westfall-Young method) 3) You can do it using Bayes, but it will be complicated and involve having to calculate the relative probability of the data given various different hypotheses (props to Jacob Falkovich for actually doing the Bayesian calculation; in case you’re wondering, the Bayes factor for my series of ambidexterity experiments is 216). Some good links from the comments: Why Experimenters Might Not Always Want To Randomize, And What They Could Do Instead, What Is Stratified Random Sampling?, Rerandomization: What Is It And When Should You Use It.
2: Comment of the week MIGHT BE saprmarks explaining why the Georgia voting reform law doesn't say what you think it says, but I'd also like to hear from people who think it says things, about whether they think saprmarks is wrong and it does say those things after all. If anyone makes a good argument for that I’ll highlight it on next week’s open thread.
3: Russell Hogg reports having an ACX-adjacent podcast, Subject To Change, which has interviewed people like Bean and David Friedman, as well as wider celebrities like Bret Devereaux, Tyler Cowen, Agnes Callard, etc.
4: After writing my post concluding Vitamin D probably does not treat coronavirus, some people suggested I make a bet with Rootclaim, who is offering to bet $100,000 that Vitamin D does cure coronavirus. I talked to Saar Wilf of Rootclaim, who was very helpful and responsive, and we had a good discussion about the evidence in favor and against. The result: Saar convinced me to shift from a 75% probability that Vitamin D doesn’t work to more like a 66-70% probability; I convinced Saar to back off from his previous betting terms that scientists would soon acknowledge Vitamin D worked better than steroids or remesdevir (not because he thought Vitamin D didn’t work, just because science isn’t self-correcting enough to change its mind that quickly or conclusively, which I agree with). We tried to come up with some other agreeable set of terms, but weren’t able to make something work given my relatively high level of loss aversion. Overall I came out of the discussion with a high level of respect for Saar, and I’d like to investigate Rootclaim further at some point.