"Either a zoonotic virus crossed over to humans fifteen miles from the biggest coronavirus laboratory in the Eastern Hemisphere."
As they say, 'you couldn't make it up'. Or, 'truth is stranger than fiction'. It does *seem* very suspicious co-incidence that a new virus that would turn into a world-wide pandemic just happened to pop up out of nowhere on the doorstep (as it were) of a specific institution dedicated to doing this kind of research, but there was no connection. And yet we seem to be forced to accept that this is so.
Congratulations and gratitude for sitting through 15 hours of video to produce this post! And I did appreciate the Linear B/Lineage B joke.
What you say about raccoon dogs here is mistaken. Raccoon dogs are not a plausible intermediate host for sars-cov-2 on the basis of information that has been known since 2021. There are several considerations.
1. Xiao et al (2021) - https://www.nature.com/articles/s41598-021-91470-2%E2%80%8B%E2%80%8B%E2%80%8B , which includes a co-author of Worobey et al (2022), a leading zoonosis paper states in table 1 that the raccoon dogs were wild caught in Hubei, not farmed as you assert in the piece. This alone rules out raccoon dogs as plausible hosts for two independently sufficient reasons. Firstly, there is unanimity in the literature that the bat ancestral virus to SARS-CoV-2 is in southern Yunnan or South East Asia. Everyone agrees with this, including Shi Zhengli. If a species was wild caught in Hubei, then there would be no explanation of how it acquired the ancestral bat virus, given that Hubei is 1000 miles from southern Yunnan.
Secondly, a mystery of sars-cov-2 is how it acquired the furin cleavage site that makes it so transmissible. There are 850 known sars-like coronaviruses, and only one with a furin cleavage site. According to private messages exchanged by proponents of zoonosis, the furin cleavage site could not have been acquired in the market because the density of animals was too low (only 3-4 per cage). When avian influenza acquires a furin cleavage site that occurs on farms with thousands of chickens densely packed, i.e. not in the wild and not when there are a handful of animals in cages in a market. https://usrtk.org/covid-19-origins/visual-timeline-proximal-origin/
2. Wang et al (2022) https://academic.oup.com/ve/article/8/1/veac046/6601809 also confirms that the raccoon dogs were wild caught in Hubei. What's more, Wang et al (2022) tested 15 wild raccoon dogs of suppliers of Wuhan markets, including the Huanan market, in January 2020 and found them to be negative for SARS-CoV-2. On average, 38 raccoon dogs were sold across the four markets in Wuhan from 2017 to 2019. So, the 15 raccoon dogs likely comprised nearly the whole inventory of raccoon dogs that would have been supplied to the Huanan market at the time.
In short, the raccoon dogs supplied to the market were tested and were negative. It is very strange that the raccoon dog narrative has persisted for this long given this public information. Even the strongest proponents of the raccoon dog hypothesis have walked back their bold claims that raccoon dogs are thost.
Xiao et al (2021) has a list of species sold at the Huanan market. I would encourage you to read that list and suggest which animals you think are plausible, and I will tell you why they are not actually plausible.
Roko (the one infamous for the basilisk) has argued that what might have appeared as a shift in expert consensus is in fact "manufactured consensus." This is worth considering. Link: https://www.lesswrong.com/posts/bMxhrrkJdEormCcLt/brute-force-manufactured-consensus-is-hiding-the-crime-of | I also believe an algorithm somehow sorting, weighting evidence cleverly could solve the problem of "too much evidence." The CIA, in fact, at least has used one such algorithm, invented by Richard Heuer.
I have no horse in this particular race, but I do have a lot of expertise in some of the areas rootclaims "investigates" (especially the stuff related to Syria and chemical weapons) - where their analysis is so shoddy and laughable it's indistinguishable from Youtube conspiracies - and the biggest surprise to me here is that anybody really bothers with rootclaim in the first place? The more you learn...
How do you account for “one of the most powerful political actors in the world which has a history of controlling information flows runs the area in question and can be manipulating what evidence is available?”
I get that you have to work with the evidence you have. When the evidence you have is passing through a filter that has pretty strong incentive to hide its own wrongdoing (if that were to exist) how does that fit into your analysis?
Did they assess a probability that, ie, certain research records where the researchers tried that ~exact~ furin site were destroyed by the CCP? Wouldn’t that just be a “naked prior”?
I enjoyed this piece - I tried to follow this story in 2020 and have always leaned more zoonosis than lab leak (I believe I said 90-10 publicly in 2021, probably closer to 95-5 now).
Despite this, I've often been frustrated and annoyed by mainstream virologists who dismiss LL with poor reasoning, and almost always find myself defending (more reasonable) LL proponents.
I guess that this annoyance has led lots of people in rationalist-adjacent circles to lean further LL than they should have. I wonder if this is a common rationalist failure mode we need to update on.
I don't think this grapples with the problems the Mr Chen case poses for the zoonosis argument. Both sides in the debate agree that Mr Chen contracted symptoms on 16th Dec. This suggests he was exposed around 10th Dec. He thinks he was exposed on the train or in hospital on 8th Dec.
This raises two problems:
1. Mr Chen lived 30km away from the Huanan market on the south of the river, and never went anywhere near it in December, according to his interviews. His case was only recorded because he happened to be transferred to the top tier Wuhan Central Hospital, which is a sentinel hospital for respiratory outbreaks in Wuhan. It is in the north of the river from his local hospital because his relative happened to work there. This is evidence of a geographical bias in he case search. How many other cases were missed on the south of the river because they happened not to visit top tier hospitals, which were largely near the market.
Geographic bias in the case search is also confirmed by:
2. Given that there was a confirmed case 30km from the market around 10th of December, this suggests widespread community transmission across Wuhan in early December and late November. i.e. not that there was a zoonosis in early Dec. The first confirmed case was on 10th/11th Dec (even for that case, it is unclear whether the first case was market linked). There was no published retrospective case search. The first case was initially thought to be 1st Dec. Usually if you do contact tracing and a case search, the first official case goes backward in time. In this case, it went *forward* in time to 10th Dec.
My final comment. It is important to consider how different SARS-CoV-2 is to other zoonoses. I have a challenge for zoonosis proponents to find me a zoonosis with all of the following features
- Spillover occurred after 2000 when sequencing became much cheaper
- There were more than a hundred human cases
- There are zero infected animals.
This characterises SARS-CoV-2, but no other zoonosis meets these criteria. Why?
Thanks for your deep and well-reasoned dive into a truly important topic. For me, your most important take-home messages were a) Bayesian reasoning has serious practical limits; b) we need to take the potential dangers of viral lab leaks seriously; and c) we need to take the potential dangers of natural zoonotic spread very seriously. One point I haven't seen mentioned is that the Earth harbors roughly 10**27 virus particles.
My rule of thumb for something like this is 'whose evidence stands up to scrutiny'. I am not smart enough to understand any of this stuff, but I vaguely knew about the evidence beforehand from podcasts and social media. And then Peter just explained it all away effectively without significant pushback. This never happens in UFO debates. Or political conspiracies. Someone spends a billion hours researching sounds smart makes 30000 claims and then you pick 5 at random and 4 are nonsense and 1 is true or misleading or misinterpreted. I notice that my plausible theories of lab leak after hearing Peters argument about the implausability are *not* those advanced by rootclaim and so I assume they are obviously wrong I just don't know why but I'm confident Peter could convince me instantly. Rootclaim's inputs fell apart under scrutiny', which makes me doubt his statistical analysis. Garbage in garbage out. Note my real confidence about who is right is super low because I am dumb. But the debate definitely moved me away from lab leak 70-30
A lesson Scott draws is that Bayes is effectively impossible to apply in complex real-world situations, and that ultimately we are forced to rely on our intuitive reasoning. We’ll just throw a little math in at the end to make sure we aren’t making some clear and obvious blunder. I suggest this means the true epigraph for ACX should not be “P(A|B) = [P(A)*P(B|A)]/P(B), all the rest is commentary,” but rather “commentary, and all the rest is P(A|B) = [P(A)*P(B|A)]/P(B).”
> When psychoanalysts claim their therapies work, they don’t mean that someone who just read a two page “What Is Psychoanalysis?” pamphlet can do good therapy. They mean that someone who spent ten years training under someone who spent ten years training and so on in a lineage back to Freud can do good therapy.
And Robyn Dawes in House of Cards showed the evidence doesn't support the idea that training makes them any better.
“One read out of 200,000,000 is completely statistically insignificant,” said Pond. “It really had no SARS-CoV-2. There is no evidence based on genetic analysis there was SARS-CoV-2 in that sample. One read out of 200,000,000 — it could have been a low level of trace contamination.”
What’s more, as Bloom’s preprint reports, Q61 was the only swab above a certain threshold for raccoon dog genetic material that contained any SARS-CoV-2 RNA at all: “13 of the 14 samples with at least 20% of their chordate mitochondrial material from raccoon dogs contain no SARS-CoV-2 reads, and the other sample [swab Q61] contains just 1 of ~200,000,000 million reads mapping to SARS-CoV-2.” When Bloom plotted the quantity of animal genetic material found in the swabs with their SARS-CoV-2 RNA content, he determined that there was in fact a negative correlation between the abundance of SARS-CoV-2 and genetic material from raccoon dogs in the swabs.
But also.... What happened to discussion of "NIH-funded gain-of-function research" by Peter Daczak and EcoHealth alliance? Is this the "DEFUSE" grant I'm question? It makes not mention of University of North Carolina - that's new to me, though of course I didn't delve deep into the grant.
I also wish it would be possible to go and collect all the Twitter posts on this topic from when the situation was unfolding, before and after the pandemic left China, including all sorts of claims: spread through plumbing, claims of corona virus circulating months (!) before the official first case, etc. -- do an analysis of how much is real, how much is fake. For example I remember following very closely everything I could read and don't remember racoon-dogs ever being mentioned, but pangolins were.
Can someone explain to me Peter’s argument "doubling rates at the wet market points at zoonosis rather than a superspreader event" ?
Intuitively, the two scenario (zoonosis / superpreader event) looks the same to me : an infected (lab worker/racoon dog) (goes at/is brought to) the market and infects market participants. I expect the same doubling rates in both cases.
Is it really such a coincidence that the outbreak occurred naturally nearby the lab? It's reasonable (I guess, I don't know) that the lab was strategically placed there in the first place due to the area's wildlife, which host many animals relevant to their virology research. If so, the alleged coincidence significantly fades.
I think the "lab leak" side of the debate focused too much on the probability that SARS-CoV-2 was engineered, and didn't properly account for the scenario that SARS-CoV-2 was a natural virus collected by virologists that accidentally leaked. In my opinion the latter scenario is more likely than the former.
The issues with extreme odds, focusing on the big picture, and eating one's losses are all related to the same phenomenon: The high complexity of the underlying processes.
Reality is not generated by a simple "zoonosis" factor competing against a "lab factor", as one would think if one naively took the simplest Bayesian models seriously. Rather there was a wet market and a lab and a bunch of people all doing a bunch of different things, etc.. Way too complex to explain. (Though because of funky math where we condition on the fact that there was a pandemic, we also can't ignore that there is a competition between these two factors; they just aren't the generators of reality, and therefore not of the evidence either.)
It is extremely common to have pieces of evidence with extremely low priors, because these pieces of evidence contain a lot of information about the specifics of these underlying processes, and probability drops exponentially with information. Notably, contra focusing on the big picture, accurately integrating this evidence requires building a narrow, specific picture of what happened, because otherwise you don't really know how to evaluate the interactions between the probabilities when conditioning on multiple things.
As you build these complex, low-probability models, that means you end up eating a bunch of losses, in rough proportion to the evidence you add to your model to account for the complexity. If your resulting model of what happened doesn't have very low prior probability, you're not thinking in enough detail.
Did either side get into the US intelligence assessments at all?
It seems to me that by far the strongest argument for lab leak is the US government conclusions to that effect. These agencies have an awful lot of raw data that no one else does, and the resources to put a lot of analytical effort into the problem. Obviously, they're fallible but they have access to the same scientific data as everyone else plus whatever kinds of useful classified raw data the US government has scooped up over the years and they employ experts on stuff. That's enough for a pretty decent presumption in their favor, especially when so much of the debate centers on data from Wuhan that could easily have been manipulated.
I'd find the zoonotic origins argument a lot stronger if someone from that side could explain what the FBI and DOE may have seen that led them to get it "wrong" based on some premise other than incompetence.
Am I the only one who Peter lost at the end when he starts asserting that any evidence that disagrees with him comes from liars? And his "evidence" that people are liars is stuff like "it was reported in the Daily Mail" and "he was a drug user" and "his cat got it too" (by which Reed obviously meant tested positive, which can absolutely happen to animals, inanimate objects, and all kinds of other random things). Followed by, "and everyone else was also a liar".
I have a friend who is convinced he caught COVID in Jan 2020 before anyone knew what it was or what to watch out for. He had symptoms that were an exact match. Officially, COVID did not exist here at that time. Is he also a liar? I assert that he is not - he has no reason to lie about this, and is an honest and upstanding gentleman. I therefore assign it a 1:10,000 chance that he is being deceptive, and thus COVID clearly spread far earlier than officially stated.
I also have a heuristic about people who dismiss evidence on the basis that it appeared in a tabloid, or wasn't in a peer reviewed papers: naive and gullible. Here's some Bayes for you: what's the probability that a journal publishes an article full of photoshopped images and garbled machine-generated text? My prior for this happening in an academic peer-reviewed journal is high, because there keep being incidents where whole editions of journals get retracted due to every article being gibberish. My prior for it happening in the Daily Mail or any tabloid is zero because it's never happened. Clearly, the Mail is far more reliable than peer reviewed journals and Peter should accept that its evidence overrules anything he's read in journals.
Rephrased: most of Peter's evidence implicitly assumes the testimony of China, virologists and epidemiologists is reliable (e.g. the ridiculously precise stated doubling period), when we know it's not. Many times he makes statements to which a good response might be, "and how do you know they aren't simply wrong/lying", but Saar was too generous to do that. Peter did not return that generosity!
Ultimately, although it was interesting and Peter's performance was impressive, I don't know if this kind of debate helps. I remain in camp Lab Leak.
As I've been saying since the whole debate started, whether or not it was a lab leak is really irrelevant. What we know for certain, with no actual controversy surrounding it, is far more interesting: China did not shut down international air travel out of Wuhan until significantly after they knew the virus was circulating.
While they were covering up the origins of the virus, and locking down Wuhan to try to keep it from spreading further internally, the CCP was knowingly, deliberately exporting this deadly plague to the rest of the world. And that is far more damning than any question of its origin. The origin of the virus is China. It deliberately inflicted it upon the rest of the world, which makes the pandemic a *de facto* bioweapon attack, whether or not the virus was developed in a lab with the intention of building a bioweapon.
If SARS-Cov-2 did not come from WIV then presumably WIV and the Chinese government would have had a very powerful motivation to open their records to outside investigators and make all of their people available for interviews. Instead they clammed up. Why?
I hadn't heard of Rootclaim before this, but that first screenshot which puts the odds of widespread fraud in the 2020 election at 8.7% tells me all I need to know -- that is orders of magnitude too high as anyone who understands how us elections are run would understand. Seems like there's something about this method, at least as practiced by Saar, that produces excessive probabilities of out of consensus or malicious explanations for things.
On the subject of this showing up baysian analysis, since viewers all tried to track it and got different results anyway:
Quant trading also works like this - it's basically using baysian inference for the stock market - and also ends up with a lot of big discrepancies and requiring a lot of Metis to get right. But it's still consistently significantly outperformed (in terms of real money earned in the real market) over traditional non-quanty trading. To the degree that baysian analysis does work in real life, this is what I'd expect it to look like.
"1/10,000 Wuhan citizens work at the wet market. So if a lab leak was going to show up somewhere random, the wet market was a 1/10,000 chance."
Why are we only counting the people who *worked* there? Wouldn't the relevant number be all the people who visited there (a much higher proportion of the population)?
First image: "The election was no different from previous elections" is not inconsistent with the claim that there was widespread fraud. As stated, none of the options are mutually exclusive.
(this is true regardless of the actual answer to the question)
I disagree with some of the arguments. I think COVID originally started spreading in rural villages. It didn't spread fast, because there wasn't that much unventilated space, and it wasn't well adapted to humans. And it wasn't detected, because until special tests were developed, COVID just looked like "atypical pneumonia". That is where it adapted to humans. Then it was carried into the wet market by a vendor. So it did spend some time learning how to spread in humans, but it wasn't by gain-of-function research.
Whichever debate participant is most well-versed is most likely to have thoroughly understood the topic and thus is most likely to have a correct intuition on the subject. Thus, we should just decide on these issues based on our subjective intuition about which debater seems to be most knowledgeable and rational, and not reason through the content of the debate except to fact-check the debaters and ensure their reasoning follows so they can't get away with intelligent-sounding bullshit. For example, instead of checking every single fact, just check a simple random sample of them. This approach may seem objectionable to some people, but it's probably what we're all subconsciously doing anyways, and it's probably approximately correct.
> this disease that came out of nowhere and ruined all of our lives for a few years
I wish we could have a debate on what the proper response to COVID should have been. In theory, if governments refused to do much other than developing vaccines, COVID would've had a significantly smaller impact on our lives. But also this would've also had significant costs to many people in the form of losing their elderly relatives and more people would've had long COVID.
I wish we could have a Rootclaim debate on what the Western countries should've done in late February/early March as a response to the virus. I think there's a large degree of disagreement about this question to this very day and it's definitely not been discussed enough in retrospective.
Awesome analysis. And really good that you picked up on the personality factor at the end. Kinda odd that Rootclaim do deep Bayesian stuff on the arguments, but do no personality screening for willingness to be objective / fear of losing a debate. The adversarial system is super-Western, I guess, with two dudes fighting their corners,. But in this instance it seems to me like it could be huge factor to swing things.
The comment at the beginning suggesting that it was too bad Putin doesn't have cancer felt a little odd to me, BTW.
You know, I looked at the "widespread fraud" one at the top, and it struck me that it didn't even consider that I would think was a much more likely hypothesis for it: That several big city political machines, acting independently and in parallel, jumped in an applied diverse methods of falsifying the outcomes, in just those places where such falsification was both (a) readily manageable and (b) critical to their respective states' electoral votes (giving motivation). It makes me suspect that you can get the results you want from Bayesian methods by carefully curating the initial hypotheses to compare. Not very different, perhaps, from C.S. Lewis's trichotomy about Jesus.
Hot take: Peter clearly failed to convince anyone.
The lab leak odds, in log10 (i.e. orders of magnitude are):
Peter -20.7
Saar 2.7
Eric -3.1
Will -2.5
Scott -1.2
Daniel -1.4
One of these numbers is clearly an outlier. Scott mentions it and calls it "trolling", I would argue that it is debating in bad faith. 2e-21 is a ratio which is just silly. For one thing, the gain of function at WiV pathway is not the only pathway towards a lab leak. The WIV could also have released a naturally occurring coronavirus at the wet market. At 2e-21 odds, we would probably have to consider the possibility that the WIV built a time machine and went back in time to infect the wet market.
Apart from the pangolin which got retconned out in favor of the raccoon dog, one thing I remember from the earlier discussions was that proponents of the lab leak pointed out that the bat with the virus likely did not come from Wuhan. Given that Saar has not made that argument, is that no longer the case? I imagine that by now we have a much better understanding of naturally occurring bat corona viruses in China.
thank you for mentioning my "preachy and annoying" blog. :P
Can't do much about my eyes rolling backward on this topic, but since you are all about evidence and probabilities; you might want to keep your eyes peeled for my next 15000-word sci comm article that comes out in two weeks and will bring people to the cutting edge of origin science, as well as put the "gain-of-function research" narrative to bed once and for all.
I was pretty sure about lab leak, now I'm 50/50. Normally I should be frustrated to end up 50/50 after reading all this but I'm filled with a feeling of satisfaction. Great job for all involved.
I am WAY out over my skis here…but it just seems like there is no objective way to actually TEST any of your priors in a scenario like this. So you’re just giving a finite number (and “false” sense of precision) to what actually remains a messy and murky quandary.
i "like" the Cosmic Joker hypothesis, meaning it's the most *interesting* one to me. i also like the idea of the Covid virus as a virus in a superposition, taking inspiration from Scott Aaronson's comment in a podcast that "There is a serious prospect that in our lifetimes people will be able to create superposition States of, let's say, a virus". (dunc tank podcast, July 11, 2020, with minor editing from his improv oral) in this krazy view, the two states of the virus are: (i) being from the Wuhan virology lab: and (ii) being from the wet market. the evidence never resolves decisively to one of them. to put it another way, a meta-version of the lab leak hypothesis is true: the Lab is some Schrodinger's Box, and the Covid virus escaped while still in its superpositional state. it moves though our world leaving a neat two-path evidential trace.
Very nice, thanks for watching, and reporting. I want to make an argument for agnosticism. It seems like we are all inclined to think we must have an opinion on almost everything. What is wrong with saying, "I don't know"? Being trained in physics, I just want more good data before I try and make any sort of decision. Eventually I hope we will hear news of some intermediate animal host, or news from some researcher that says, yeah we were doing GOF on those viruses. Until then I don't see how the odds (for me) can be anything other than 50/50. A sign of my almost complete ignorance*, and an unwillingness to give anymore credit to one sides 'experts' vs the others. I think the best path for humanity is to assume both things are true. And try to make both forms of possible virus mutation less likely to happen in the future.
Oh, and why are there no error bars in all of this analysis. I'm thinking the error bars should be huge.
*I read a lot of stuff about this in 2020, said then we need more good data, and stopped spending time thinking about it.
Thanks for the write up Scott! This is definitely very interesting.
I find myself walking away with a couple of thoughts. First, about the debate itself:
1) I started out pretty strongly leaning towards but far from certain about Lab Leak being correct. A lot of this I think stems from finding out that the anti-lab-leak side was a literal conspiracy that had a lot of institutional support to try to shut down the debate. Even those shouting down the theory secretly thought it might be true (at least early on)!
2) Likely related, I could feel learned helplessness setting in while reading the back and forth descriptions of the debate. With no priors on the believability of either participant, and seeing that both were making very strong claims that could reasonably answer their opponents, I want to mentally shut down and hold my existing perspective.
3) I end the debate slightly closer to zoonosis/no opinion, but still leaning towards lab leak.
Meta-discussion I feel much worse about Bayesian reasoning, especially the hard version with math. Peter clearly has a number that's beyond implausible, but the error bars on all of the numbers seem unworkably large. You could adjust all of the factors up and down by relatively small amounts and come up with any interpretation.
Even more, sometimes really really implausible things happen. As Scott notes here, something really implausible definitely did happen! In fact, across the world, implausible things happen quite often. People die of incredibly unlikely events all the time - an average of about 7 kids per year in the US have drowned in buckets since 1984. If you had never heard that number, what probability would you put on the chances of a child drowning in a bucket? There are 22.4 million kids 0-5 in the US right now, with seven per year drowning in a bucket, which suggests about 0.0000003125 chance of a child drowning in a bucket in a given year. No matter how you do the math, the chances are very truly low. But it happens seven times a year.
How would we model COVID's origins if something incredibly unlikely happened? It's not impossible that it originated in neither the Wuhan Institute nor the wet market. In the realm of possible, it may not even have started in Wuhan. Saying that something is *more* or *less* likely is all well and good, but reality happens in exactly one way, which can be the very rare event. I think 1/10,000 is incorrect for the likelihood here, but sometimes 1/10,000 things happen. Just because we say something is unlikely doesn't mean it's untrue. Saar implicitly believes that the wet market theory is ~6% likely to be true. 6% is far from ruling something out, even if he's arguing as if it's closer to 99.99% the other way.
#2 My beliefs shifted from pretty confident in lab leak to 50-50
#3 Reading through your summary, I often got this horrific feeling of college debate and dropping points. College debate has this concept of dropped arguments, where the worst thing you can do is not respond to a specific argument your opponent makes, because they automatically win that argument. That's what this felt like, I'm not actually convinced by Peter's overall story, I'm not even sure there's a coherent story there, but Peter kept coming up with specific situations where Saar was wrong and that convinced me.
I'm reading back through the lineages thing and, just reading through, Peters positive argument for why the older lineage started spreading later is pretty handwavy. Compare that to the next section where he goes through why the intermediates are a programming error, he's on point. But I come back to the core question of why the older lineage started spreading later than the new lineage and I don't have a great answer to the main point, and I think Saar's is better here, but Peter dominates on the subpoint.
I'm just scared of focusing too much on the subpoint and irrationally weighing that too heavily. I dunno? For people who watched it, did you get "debate bro"/Destiny vibes from Peter? That's what this kinda smells like, I notice intense debates should work but I could also watch 50 debate streams and at the end of it be dumber than I started and I felt this way a lot in college debate as well.
This was a wonderful read and personally seems like a great experiment. A bunch of credit goes to Saar and Rootclaim for courtesy and making this public and educating all of us, whatever position you hold.
An insanely good debate in many ways, and congratulations to Scott on a most excellent summary. This reads like a communication from an alternate universe. Seriously, since when are there 15-hour highly structured debates at the object level with this level of intelligence brought to bear? That just does not happen in this world, except it just did.
Despite the outcome not being what Saar wanted, he has ably promoted Rootclaim as a useful tool. I want a lot more of whatever this is.
Despite Peter being one of the smartest people I've ever encountered online, arriving at an odds ratio of ~10^21 against lab leak is prima facie evidence that he has not correctly employed Bayesian reasoning. That sort of ratio is where you start getting "simultaneous miracles". I estimate that the chances of a *natural* zoonotic event, meaning one in which human action was not a causal factor, infecting its first patient *inside the virology lab*, as being *considerably* more likely than 10^21 to 1 against. I possibly could estimate the chances that I, personally, created COVID, had it released at the Huanan Seafood Market, and suppressed my memory of doing so, as 10^21 against. (By this I mean if I sat down and calculated how likely such an event was, 10^-21 is an answer I might obtain. In reality I would throw out any probability that low and group the event with "weird shit happens".) 10^21 is a sextillion. I can't count that high and neither can Peter.
Not getting answers like 10^21 touches on a real problem with practical Bayesian reasoning, which is that almost all independent events are only very nearly independent. Clinamina like the Self Memory Wiping Garage Biolab Operator Who Is Me exist in enough possible worlds to start having an effect if you multiply together enough large Bayes factors. This is far from disastrous to the project of being better Bayesian reasoners but there should be more effort going into developing and teaching the techniques that deal with it. Rootclaim looks like one of the places that might happen.
During the week of January 23rd, 2020, the day lockdowns were enforced, there was somewhere between 175-225 excess pneumonia deaths in the Wuhan DSP per 100K. This is eyeballable in figure 1 here.
They report 243 per 100K for the entire first quarter of 2020. That's an *enormous* number.
Smooth this out over a week and say that during the week of 1/23/2020, ~30 more people than expected were dying from pneumonia in the Wuhan DSP per 100K.
The cool thing about the beginning of pandemics is that there's no immunity and the mixing assumptions of SIR models probably work well in the beginning of a respiratory outbreak. You can play with this one:
I couldn't find good information on the size of the Wuhan DSP, but allow it to vary between 2M and 10M people. Assume the R0 was between 3-4, the CFR was .5%-1%, and a few other now fairly well known things about the course of the disease. And look at the fatality numbers.
Using these ranges and standard compartment model logic, I don't see how you get such an enormous increase in excess pneumonia deaths consistent with an early December outbreak in the HSM. It doesn't make any sense at all. The best case scenario from these simulations is sometime early in November 2019.
Several of Peter's arguments treat the doubling rate as a real property at low populations. That doesn't seem right.
If patent zero only infects one person, the pandemic is delayed by one doubling period. If they infect 4, it arrives one period sooner. What are the relative odds between someone infecting zero, one, two, four, or fifty? When there are enough infected this kind of effect averages out and we can just say it doubles at some rate. At small numbers you get swings in infection progress based on how each person rolls.
Because of that attempts to extrapolate infected populations backward using the doubling rate seem doomed once you get into single digits. The error bars on that kind of estimate should be large. It's noisy.
So I don't think you can confidently say things like this
> The COVID pandemic doubles every 3.5 days. So if the first infection was much earlier - let’s say November 11 - we would expect 256x as much COVID as we actually saw
Or this
> Although Lineage A is evolutionarily older, Lineage B started spreading in humans first.
> We know this because Lineage B is more common. Throughout the early pandemic, until the D614G variant drove all other strains extinct, a consistent 2/3 of the cases were B, compared to 1/3 A. Both strains spread at the same rate, so the best explanation is that B started earlier than A. Since COVID doubles every 3-4 days, probably Lineage B started 3-4 days earlier than Lineage A, which explains why it’s always been twice as many cases.
> This is a pointless too-clever-by-half “flourish” that there would be no reason for a human engineer to do.
Have you ever met an engineer? We do this kind of nonsense all the time! This particular argument cuts in exactly the opposite direction Peter intended.
I don't have any comment on the outcome, but this little detail drove me nuts.
"As per last year, we will not be subco ntractin g any funds to the intuiti ons in these
countries. All efforts expended in these co untries will be from collaborating
partners and not funded by our award. PI, Co-Investigators or other team
members may con duct short field ttips to assess markets, identify wild life in
them, and anange for shipment of samp les of bats and other high-risk host species
in co untri es that nei ghbor China (Burma, Vietnam, Cambodia, Laos) and that
supply wildlife to the internationa l trad e to China (Tha iland , Malay sia,
Indon esja)."
i.e. they say in 2017, they did sampling in these countries, and all samples would be sent to the WIV. Weirdly, all of the closest known relatives to SARS-CoV-2 have been found in places the WIV was actively sampling for more than a decade.
This is not even considering the sampling that the WIV would have done without EcoHealth funding.
The DEFUSE proposal proposes extensive bat harvesting in Yunnan in 2018. This is also evidence against them having given up on bat harvesting in 2015.
My intuition is that Rootclaim is correct that you should be able to use Bayesian spreadsheets--especially after some adversarial collaboration--and get a format that consistently outperforms intuition and even professionals informally weighing the evidence. It's sort of like how as an analyst I use fermi-calculations to get much more reasonable estimates of unknowns than just trying to estimate the unknown directly. I think Rootclaim should focus more on testing that.
I'm with you (after reading two long summaries of their debate) that zoological is more probable but also that Rootclaim is a worthy project. I wonder if the change in format that Saar should focus on though is more emphasis on the spreadsheet and adversarial collaboration to get both participants using the same spreadsheet. I'd love to see a version of the spreadsheet that two smart opposing viewpoints people agree is a reasonable setup (without any multi-stage fallacies, etc.) so we can see which line items are driving most of their disagreement free of any logical reasoning errors (or at least free of errors that adversarial smart people could identify). And then judges can focus more on the individual ratios (and any remaining spreadsheet steps that debaters couldn't adversarially reconcile)
That was really interesting. My bayes argument would be something like "If covid WAS leaked from the Wuhan research lab, how plausible is that picture of events?" If so I'd expect an average amount of "well, we don't know" for scientific research. But each thing that needs to be explained away rapidly adds to "that doesn't sound right". Today I learned that they did do modifications like this. But I also learned that they were usually publishing their research like any academic institution, so you need coincidences or cover-up to suppress anything they'd done on COVID specifically.
This includes a lot of things that I don't know. But I still like my reasoning, because I know there's lots of biologists out there who have a better picture than me. And I haven't heard of any reason for biologists to avoid thinking about a lab leak, and don't think it's likely there could be a reason that wouldn't be well-known. (Whereas there ARE reasons for physicists to dislike many worlds, or doctors to dislike "chronic fatigue exists and easy weight loss doesn't") But what I usually hear from random biologists is "well, yeah, it could be, but I don't have any particular reason to think it is". And so I think that's probably right.
I'm interested in the ways of putting more specific numbers into reasoning but I think you can rapidly get into territory where it's hard to tell if the reasoning is sound. E.g. trying to describe how much of a coincidence each theory is raises questions like, if it was zoontic, how many other places would have sounded as suspicious as the wuhan research lab? Restricted secret areas? Other kinds of labs? How unique is the nearby lab, in advance? If it was a leak, how many places would have seemed as suspicious as a wet market? Nowhere else seems very suspicious, but "coming out of nowhere" would have seemed completely plausible for zoonotism. Or how much are the lab and the pandemic both there because there's convenient reservoirs of coronoviruses nearby, that could be, I've no idea. I think trying to make these precise leads to "things we didn't know we needed to account for" swamping the analysis.
I ought to read how the probability calculation is actually working. I would be really interested. Although I'm not sure it would help: e.g. I'm more persuaded by my argument of listening to biologists, so any attempt to quantify likelihoods for genetic arguments will be swamped by the chance of something I don't know mattering.
Perhaps this description of the debate wrongly magnifies this, but it struck me as really discrediting to the lab leak side that they kept relying on debunked claims. The LL side would bring up things lime the 90 missing cases, the daily mail guy, or the idea that the market was a superspreader event, and Peter would describe that evidence in greater depth and be able to pinpoint why that evidence was faulty, which was not rebutted by the LL side. One of the most annoying habits conspiracy-adjacent thinkers online have is to gish-gallop between weak evidence they're only lightly acquainted with, and then not change anything about their thinking when a piece of evidence is disproven. I generally find the meta-bayes "what about the odds your argument is secretly bad and mine is secretly good" mostly annoying, but how can you not apply a pretty heavy discount to the credibility to a side that is frequently trotting out rather flimsy evidence?
The $100K prize introduces a huge signaling hazard, mainly because Saar is rich and Peter isn't. I'm sure Saar's lost that much playing poker. For Peter, this is a sizable portion of his net worth — so he plays to win (and, as other commenters have pointed out, may be playing a bit dirty).
That said, the biggest issue with this debate format is that it's about an *incredibly important topic* where several major players have *a lot* of skin in the game. As much as this is a battle of evidence, it's also a battle for narrative (see Peter's rationale for wanting to get one up on the conspiracy theorists). And the "Official" narrative, that all good, sensible, educated people should agree on (you don't want to be a conspiracy theorist, do you?) is zoonosis. There are probably better, less charged questions where the Rootclaim Bayesian approach would work.
Not that I'm entirely convinced by Lab Leak either - maybe 60% odds? But given that I read ACX while goofing off at work, I don't have time to watch 15 hours of debates. Gotta go with my gut at some point.
Two big issues with trying to formalize Bayesian reasoning is to properly identify what is already part of your priors and what isn't. Second, using it at scale in a systematic way so that you actually benefit from the correct updating mechanism. After all, the more you just rely on the prior the less it makes a difference.
The problem is that we only pay attention to issues once something makes us think there is an interesting question there and often these issues are relatively unique so we have no obvious choice of base rate to use as our prior. And there is the very real danger of pulling in information that is really only a result of the update.
I think distinguishing the prior is one place where AI can sorta help. Maybe not yet, but a great feature with AI is that you can remove data from it's knowledge and see how it would judge things. I'm particularly hopeful for this in politics (this AI predicts your judgement in these political issues 90% of the time and when it doesn't know that Trump supports this policy it predicts you will too).
I'm more skeptical about the ability to do this systemically at scale in a way that uses enough interconnections to improve judgement.
I think people get caught up about the furin cleavage site mutation because they're looking at the wrong base rates. To think about the furin cleavage motif, you need to ask (I'm phrasing these questions a bit imprecisely):
- How often are furin cleavage motifs studied in places other than Wuhan? How often are they studied in Wuhan?
- How often are furin cleavage site insertions the cause of viruses learning how to become pandemic? How often are other mutations the cause of viruses learning how to become pandemic?
I don't see people asking those questions. The fact that people don't ask those questions makes me suspicious of everyone involved, honestly. These appear to me to be the questions that arise out of applying Bayesian reasoning straightforwardly. (I'm also suspicious, as a sidenote, because the question overall is kind of irrelevant: everyone agrees lab leaks can cause pandemics, and everyone agrees zoonosis can cause pandemics, and everyone agrees zoonosis is more common than lab leaks, so we should just try to prevent both, focusing more on zoonosis than lab leaks.)
Also, WTF do people care so much if this particular disease originated from a lab leak.
We have a pretty good grip on the problem of biolab safety (eg the leaks in the UK) and the existence of gain of function research. So whether or not COVID itself happened to be a lab leak isn't reason to update too much on the future probability. Any forward looking policy question shouldn't really depend much on whether COVID was or wasn't a lab leak.
I get the debate is interesting but it's ironic that a claim to be more rational is being tested via a subject that it's irrational to be so concerned about.
"Either a zoonotic virus crossed over to humans fifteen miles from the biggest coronavirus laboratory in the Eastern Hemisphere. Or a lab leak virus first rose to public attention right near a raccoon-dog stall in a wet market."
One thing that strikes me is that there is an asymmetry here. The first part of the above does indeed sound like a grand coincidence. It's not like the mutating virus would know that it happened to mutate near a lab. However, the second part is only a grand coincidence if we assume that if there was a lab leak, that it was a) fully accidental, and that b) no one acting in bad faith had the ability to distort data from early testing, etc..
I'm not sure I'm of the opinion that lab leak is thus more likely. But I can't help but think: it sure would be convenient if you were a lab researcher/administrator/related government official and you didn't want to be blamed for something you negligently made possible - then it sure is convenient that there is that nice big wet market nearby that perhaps you can shift the blame toward.
My current thoughts: though I didn't sit through the 15 hours - I am now of the opinion that the zoonotic hypothesis is stronger than I had thought before. I am updating to be less confident in the lab leak. But I'm probably now to about even odds, fwiw.
Michael Weissman's analysis is by far the most principled and persuasive thing I've read on the topic, it deserves much more discussion than just the link.
The claim that you shouldn't have high Bayes factors on both sides of a debate is just false.
Every event with sufficient detail involves a ton of unlikely stuff happening. Every time you flip 30 coins you get an exceedingly unlikely list of heads and tails.
Sure, if you understand the data to be the **complete** specification of the even then yes you don't get high Bayes factors on both sides because you only get one. But that's not what's going on here. They are both cherry picking the aspects of the situation that most favor their hypothesis so of course you get high Bayes factors on both sides.
I don't have anything to add on the COVID origins topic, but I do want to register some (possible?) disagreement with this bit:
"For example, suppose I win the lottery, I’m told I win the lottery, the lottery company gives me a big check, I cash the check, and I become rich. Given that there were 1-in-100-million odds against me winning the lottery, the lottery company giving me the check and so on must be at least 100-million-to-1-level evidence - otherwise I should refuse to believe I really won the lottery, even as I enjoy my newfound wealth!"
The quip-level counterpoint: You *shouldn't* believe that you won the lottery, because which is more likely -- that I won the lottery and cashed the check and got all this claimed evidence, or that I'm *dreaming* that I won the lottery and cashed the check and all the rest? After all, I've had several dreams where I won the lottery before, and I can get just about any evidence you like in a dream (all made up by my brain, of course). Sure, a 10^8:1 update is possible in this case (er, maybe, I'm not sure because it sort of hinges on things I think I know about dreams but which might be false -- after all, I might be dreaming I know them), but the strongest counter hypothesis is this sort of out of band thing (dreaming, or maybe something like solipsism or a "living in a simulation" or "being in The Matrix" or similar roughly-equivalent thing) rather than something like an elaborate prank or a mistake.
Yes, on at least one occasion I have realized that I was dreaming by applying this logic.
This might seem academic or silly -- how do I **really know** that I'm not dreaming my whole life, being continually deceived by Descartes's demon, etc. -- but I'm pretty sure that similar caveats ought to apply to trying to reason about some things that people care about, like "Is there a God?". If you think you are more than 99% sure you know the answer based on Bayesian reasoning, you are being overconfident that you (a) understand the various conditional probabilities involved and/or (b) have properly considered all the relevant hypotheses.
I don't want to divert attention from the debate, but since it seems appropriate, I'd like to share a post I wrote nearly three years ago proposing an hypothesis that is not exactly lab-leak, though not incompatible with it, and not zoonosis (at least as I've seen it stated):
The propagation of Covid-19 seems to point to certain areas of the world being less affected by the disease for no particular reason (no younger population structure, no less obesity, etc...). The main area being Southern China and Northern SEA. This is precisely the area where you would expect zoonosis for a bat coronavirus to happen. Other geographic areas correspond to places where infected people might have first, and in greater numbers, arrived if they had been infected by a previous virus in the aforementioned main area. This and other evidence (see full post) point to the possibility that a previous ancestor (to SARSCov2) virus emerged and propagated in that area, before mutating into SARSCoV2.
The competing 1/10,000 claims about the wet market are interesting, but feel too speculative without enough hard evidence for me to know what to do with. Far more salient and intriguing to me are:
Saar's claim that the 12-nucleotide furin site comes out of nowhere...
> COVID - which mostly just resembles BANAL-52 with a few scattered single-point mutations - has twelve completely new nucleotides in a row - a fully formed furin cleavage site that came out of nowhere.
vs. Peter's claim that the the furin site doesn't use the expected sequences for such a site and actually uses sequences that wouldn't be expected to work. Plus Peter's claim that WIV didn't have the technology or expertise to insert such a site anyway
> COVID’s furin cleavage site is admittedly unusual. But it’s unusual in a way that looks natural rather than man-made. Labs don’t usually add furin cleavage sites through nucleotide insertions (they usually mutate what’s already there).... COVID’s furin cleavage site is a mess. When humans are inserting furin cleavage sites into viruses for gain-of-function, the standard practice is RRKR, a very nice and simple furin cleavage site which works well. COVID uses PRRAR, a bizarre furin cleavage site which no human has ever used before, and which virologists expected to work poorly. They later found that an adjacent part of COVID’s genome twisted the protein in an unusual way that allowed PRRAR to be a viable furin cleavage site, but this discovery took a lot of computer power, and was only made after COVID became important. The Wuhan virologists supposedly doing gain-of-function research on COVID shouldn’t have known this would work. Why didn’t they just use the standard RRKR site, which would have worked better? Everyone thinks it works better! Even the virus eventually decided it worked better - sometime during the course of the pandemic, it mutated away from its weird PRRAR furin cleavage site towards a more normal form.
Peter's claims seem stronger to me, and I didn't see any response to these specific claims in the summary (although I may have missed), but I also can't explain Saar's claim, nor do I have the expertise to assign probabilities to any of this. But these claims seem to me to be the most important and what I would love to see further discussion about.
I just wanted to respond to this point in the conclusion:
"Cool Machiavellian plot you have there, but maybe the fact that you’re losing 16%-66% should make you question whether you’re really as smart as you think you are...At some point you have to start debating!"
The reason that people in the field might be reluctant to accept this point is that they don't, and shouldn't, give a shit what 66% of the American public think. And debating just gives the American public the impression that it has some kind of a right to have a say in what the truth is. If you're a virus researcher now, working on how to predict and prevent the next zoonotic infectious epidemic, you really really don't want the public deciding to shut you down or order you to focus on lab leak prevention.
This is obviously very problematic - I'm kinda making the argument for important things to be decided in smoke-filled back rooms. But it's also unavoidable, I think. Science must by its nature be difficult and cutting-edge and unknowable to most people; and scientific advances are by definition decided by something other than human opinion - ideally, by what the universe is really like. So there is just an inherent limit on the level to which science can be democratised. Which means that noting which way public opinion blows is unlikely to be very relevant to the people who want to advance science.
That said, I do think that science is getting more democratic, and that that's a good thing. It's just a messy process.
I'm against using a text format for future long-form debates like this.
I watched this debate an a second monitor while doing other non-cognitively-demanding things like working on design projects and playing video games- and I enjoyed it a lot more than the podcasts and audiobooks I usually use for that purpose. In contrast to those, nothing about the debate felt like dull filler, and yet, unlike most really demanding books, it was possible to miss bits here and there without losing the overall thread of the argument. It's almost a perfect format for content to listen to while slightly distracted, and I think that for that reason, debates like this can provide a ton of value to people beyond just shifting their opinions on the topic.
Granted, a text format would probably result in slightly better arguments- though I don't buy that Peter ambushing Saar with new claims was a major factor in his loss; the lab leak side had plenty of time to research claims between debates, and plenty of time to respond later. A book-length written debate, however, is something I expect almost nobody would be willing to read, since it doesn't provide that extra value as good content to consume while doing other things. It would also require a lot more effort on the part of the debators, so I'd expect there to be both fewer long-form debates and much less engagement- and I don't think a slight bump in quality would at all make up for that loss of value.
Fantastic post - I've shifted my opinion quite a bit in response.
One thing that sticks for me is this: it didn't take long to find the civets responsible for the earlier SARS outbreaks. Why has this proved so challenging here?
The troll in me wants the answer to be that it was zoonosis that occurred in a lab, and then it leaked out of the lab.
Thank you for this excellent post. It's one of the best things I've read on the internet ever.
I remain in the Lab Leak camp, where I started. I'm of course not 100% certain, and I don't think it's possible evidence would ever emerge that would make me 100% certain. I really don't like people who claim Lab Leak is a "conspiracy theory" and am inclined to discount everything they say.
A question:
Should we accept claims by the Chinese Government and technical/academic/policy experts in this field at face value? Aren't they all hopelessly conflicted? Even if what they say is logically sound, aren't there myriad ways in which they could be fudging the analysis a bit, even unknowingly? What value have researchers doing gain-of-function work on viruses provided to the world? What is the likelihood someone in the field is going to publish findings saying the risk reward in their field just isn't worth it and all their skills are worthless because the work should be shut down?
Another question:
What do those maps of the outbreak in the wet market purport to show? The location where someone was exposed to COVID (how can that be known)? Some point that a person who got infected passed through (what about all the other locations they visited)? Whatever they claim to be showing, how accurate could their measure be? Based on interviews? How hard would it be to fudge the data a bit - for example, an infected person tells a researcher all the places they've been in the last week, and of the dozens of locations the researcher chooses the wet market to record.
First of all- awesome post! I loved all of this. Saar for setting up this challenge, Peter for taking him up on it, the judges for being fair, and you for summarizing the results- all great stuff.
As for the weird coincidence of the Wuhan Institute of Virology being located so close to the epicenter of the breakout- is it? Everyone knew that wet markets were a potential risk center for spreading diseases. See for example: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7141584/ about the 2003 Avian Flue outbreak. That's the same year that China approved construction of a biosafety level 4 lab at WIV.
They built the lab there because it was a good place to gather research data, and maybe also because the local government was strongly interested in that sort of research to protect themselves. It's the same reason that the "Center for the Study of Active Volcanoes" is in Hawaii, the "Earthquake Engineering Research Institute" is in California, and the "Hurricane Resilience Research Institute" is in Houston. Of *course* they are there where the threat is greatest. And gain-of-function research was just the most effective means of researching that kind of virus, if you want to research it before it actually starts spreading.
You might ask, if they knew the threat of wet markets, why didn't they just shut them down? My understanding is that China doesn't have that kind of state power. There are too many poor, rural people who really want to eat meat, and still can't afford ranch-raised animals. If you try and shut down the wet markets they'll just start selling them in secret someplace even worse.
Nitpick on that poll asking Americans if they think lab leak is true. (Before this) I'd have had to answer this as "probably true" but my probably was that I was at like 60% lab leak and 40% zoonosis. 40% chance of zoonosis is still pretty dang high and doesn't really seem that indicative of acceptance of the bad epistemic edifice that accreted around lab leak. I mean 30%ish saying it's "definitely true" is a problem but "probably true" seems to be doing a lot of work.
Also, good post. Quite markedly shifted my opinion.
*Edit* Also, both these posts and that parliamentary procedure kid that's been mentioned so much recently has made me think that the best model for thinking we have isn't heuristics or Bayesian reasoning or the wisdom of crowds. It's probably just smart guy who does like a ludicrously unreasonable amount of homework.
> Zero COVID era Chinese outbreaks were concentrated in wet markets because they received infected animal products. We know why there was an outbreak in the Xinfadi Market in Beijing: it was because the seafood stall got frozen fish from some non-Zero-COVID country, the fish had COVID particles on it, and the vendor got infected and spread it to everyone else. Something like this is true for the other Chinese wet market based outbreaks we know about it. So this makes the opposite point you think it does: wet markets start outbreaks because there are infected goods being sold there. Then the virus spreads through the wet market at a completely normal rate.
I definitely know that China ascribed COVID outbreaks to things like "the person in the plane on the previous flight left covid residue". China was desperate to explain every outbreak so they could continue attempting to achieve a horrible and impossible goal. I don't actually believe that is how the virus was spreading? In particular, surface contact spread and the idea that you should wet wipe your boxes quickly fell out of style. I think the ubiquity of spreads getting first observed at wet markets during covid zero is strong evidence that even if covid is zoonotic but first entered the population by like, bats during a vacation, you would still have first observed it in the wet market.
I filled this out with others' hidden, and ended up weighing that even less than Saar did, which probably means I'm overly dismissive.
I hope this isn't a derail: In one of the previous ACX posts' comments, someone posted a link to a page explaining the case for gain-of-function research. I'm embarrassed that I didn't either follow the link or bookmark it. Does anyone have that link? Many Thanks!
Way back in 2020 there was an article https://www.independentsciencenews.org/commentaries/a-proposed-origin-for-sars-cov-2-and-the-covid-19-pandemic/ , which I read after George Church tweeted it (!) (without comment or explanation). Their proposal (they call it "Mojiang Miner Passage" theory) in brief was that it WAS a lab leak but NOT gain-of-function. Rather, in April 2012, six workers in a "Mojiang mine fell ill from a mystery illness while removing bat faeces. Three of the six subsequently died." Their symptoms were a perfect match to COVID, and two were very sick for more than four months.
The proposal is that the virus spent those four months adapting to life in human lungs, including (presumably) evolving the furin cleavage site. And then (this is also well-documented) samples from these miners were sent to WIV. The proposed theory is that those samples were put in a freezer at WIV for a few years while WIV was constructing some new lab facilities, and then in 2019 researchers pulled out those samples for study and infected themselves.
I like that theory! I’ve like it ever since 2020! It seems to explain many of the contradictions brought up by both sides of this debate—it’s compaible with Saar’s claim that the furin cleavage site is very different from what’s in nature and seems specifically adapted to humans, but it’s also compatible with Peter’s claim that the furin cleavage site looks weird and evolved. It’s compatible with Saar’s claim that WIV is suspiciously close to the source of the outbreak, but it’s also compatible with Peter’s claim that WIV might not have been set up to do serious GoF studies. It’s compatible with the data on differences between COVID and other previously-known viruses. Etc. (As I understand it.)
Old as this theory is, the authors are still pushing it and they claim that it’s consistent with all the evidence that’s come out since then (see https://jonathanlatham.net/ ). But I’m sure not remotely an expert, and would be interested if anyone has opinions about this. I’m still confused why it’s never been much discussed.
This was incredibly informative. Thanks to the participants for doing it, and to Scott for summarizing and publicizing it. I hadn't thought about COVID origins in a while, but I would have told you it was in the tossup zone, and digging into this a bit puts me closer to Scott's 90-10.
I wish this was a highly replicable method for illuminating tricky topics, but I kind of suspect that the conditions here (two reasonable and sincere people who believed an impartial jury of smart, curious neutral observers would be won over to their side, and who were able to agree on who constituted a smart curious neutral observer) are tough to arrive at. I hope I'm wrong!
Not to be too facetious, but perhaps Rootclaim should have calculated the probability of the existence of a debate savant before building an attractor for one.
The big problem is that we still have very few details about all this. How is that possible after all these years?
The Chinese authorities haven't been very transparent. Other governments haven't demanded transparency from China. Why not? They just accept that China doesn't want to give any details.
The pandemic was a global catastrophe. It killed a ton of people and caused enormous economic losses.
The people deserve to know how it all happened. Secrecy is unacceptable.
The main reason why I'm leaning towards the lab leak is the severe censorship about the topic. That came from the individuals who strongly supported the zoonosis hypothesis. Usually people who desperately want to hide something have something to hide.
“Hey, this bat we just collected on one of periodic field trips to yunnan had an interesting viral strain with some really interesting structure around how it’s furin site works and implications for protein folding”
“We should probably research it, is there any grant money we can use?”
“I think the Americans just announced some new initiatives?”
“Great, let’s apply. Write a proposal that says we will investigate Furin site placement and it’s impact on virology.”
….
“Did we get the funding?”
“No”
“But the grant reviewers didn’t know about the bat, right? Just that we proposed doing the furin site stuff?”
“Yea, we’ve changed the names of the ones we found and generally obfuscated that we have this specific one in house”
“Should we just do the research anyway?”
“… … … yea, probably, could be interesting”
“We haven’t really done this before, and maybe we don’t have the right safety procedures and protocols, specifically to make sure it doesn’t escape?”
“No. But this particular strain can’t even last that long in the lab on its own. So what’s the worst that could happen”
I have no strong opinion about the Lab Leak hypothesis. I have a very strong opinion that Bayesian reasoning does not, in practice, help with deciding such matters.
Early in the post, Scott offers the following. After reading the entire long post, I believe that it is, in fact, the correct overall summary:
> But the joke goes that you do Bayesian reasoning by doing normal reasoning while muttering “Bayes, Bayes, Bayes” under your breath. Nobody - not the statisticians, not Nate Silver, certainly not me - tries to do full Bayesian reasoning on fuzzy real-world problems. They’d be too hard to model. You’d make some philosophical mistake converting the situation into numbers, then end up much worse off than if you’d tried normal human intuition.
Disclaimer: I have not watched the full videos. Apologies if this point is addressed at length.
To what extent are both parties relying on CCP-curated information for their data on the early cases? Was there a discussion of how (un) trustworthy that information might be?
If you assign 1/10,000 probability of that particular wet market being the location of the first major outbreak conditional on the lab leak hypothesis, you should also calculate the probability of that particular location under the zoonotic origin hypothesis. The population of Wuhan is <1% of the population of China (maybe the whole world is a better reference class, though rural areas and developed countries are going to have lower risk), for example, and that’s before you account for the fact that zoonotic origin could also cause major outbreaks at other locations within each city. All in all I’m not sure “covid began to break out at this particular wet market near WIV” is terribly strong evidence one way or the other.
Another piece of evidence I don’t see discussed here is the lack of openness from China, and relatedly the lack of evidence of a zoonotic source. If China had information against the lab leak hypothesis they would likely release it, so the relative lack of information we have is more likely given a lab leak imo.
For all the talk of Furin cleavage sites, there's one thing that confuses me. Could we have had a (far less deadly, probably not even noticed) coronavirus pandemic if there was no Furin cleavage site? My understanding of gain of function here is that there are multiple proteases that can do the cleaving even without Furin, but Furin does the job faster, and therefore sometimes outruns the immune system in a way COVID wouldn't do without that extra cleavage site. But that raises the question of why we got an outbreak of a virus with that Furin cleavage site, instead of a hypothetical precursor that lacks it. The existence of a Furin cleavage site is considered unusual, so for Zoonosis would you expect a precursor virus without it to be far more likely to cross over into humans first, and therefore prevent COVID from being a major issue in the first place?
The notorious fact is that there were only 3 labs in the world at that time doing gain of function technology in the world...2 of those were im US and 3rd one in Wuhan..GOF was banned by Obama administration so the only destination to outsource technology was wuhan lab...🤗🤭 even famous defuse grant for the almost same virus creation by GOF was proposed and refused for wuhan lab by darpa im 2018...immpossible coincidencenes happen😅🤣
"Still, it’s awkward to use “conspiracy theory” as an insult when the conspiracies were real. Maybe a better question is whether lab leak is “pseudoscience”.
The argument against: lots of smart people and experts believed it was a lab leak. There were all those virologists giving 50-50 odds in their internal conversations. Even Peter says he started out leaning lab leak, back in 2021 when everyone was talking about it.
The argument in favor: since 2021, experts (and Peter) have shifted pretty far in favor of zoonosis. They’ve been convinced by new work - the identification of early cases, the wet market surveys, the genetic analysis. "
I have admittedly not looked into the details, and I cannot claim to truly know how to handle filtered evidence, but:
If the experts (mostly) all agreed to have a conspiracy pushing for A, and then all the evidence that turned up after that pointed to A, this really doesn't make me feel like I have to raise my probability of A all that much. For all *I* know for every published result supporting A there are 2 unpublished results supporting B, or claims that have not been investigated because experts (correctly) believe they would turn out to support B.
One issue I have with the opening presentation in the first video (I have watched 38 minutes so far) is that he lists, for example, about a dozen different theories about how the virus could have been made, and his slide says, "it's not possible for all these theories to be correct." So what? If there's one true explanation for something, the fact that 11 other people came up with a false explanation for it doesn't diminish the probability of the first explanation being true. Otherwise, I could make the probability of any true idea go down just by creating a bunch of false theories.
Skipped the whole lot and am working purely from Scott's post-transcript summary.
>So the real p(wet market|lab leak) isn’t the 1/10,000 chance a pandemic arising in a random place hits the wet market, but the (higher?) probability that there’s something wrong with Peter’s argument.
This looks like an excuse to assign whatever numbers you like to any point. If you're readjusting part of your stats, surely you should be readjusting all the other parts with the same readjustment metric, and thus reach a conclusion indistinguishable from the original formula.
It does sound like Saar lost mostly because he got caught flat-footed, but considering he set up the system it still looks bad for his methodology. Surely he used it to calculate a proper debate setup beforehand, and it underperformed.
I learned more about this debate and the arguments in favor of each side in the hour and a half reading this post than I did in any previous discussion of these arguments. Thank you for the rich summary, and while I agree with your arguments about why this kind of thing probably can't happen very often, I really wish we'd see more of these kinds of rich debates
A systemized version of this plus community notes is my basic model for how all news should work. But with reputation scoring in there to smooth out some of these transaction costs for actors who have proven themselves credible.
“Dumb” question that I did not see answered anywhere, and not mentioned much at all even in these comments to date: does “lab leak” also incorporate the possibility that it was created in a lab and *deliberately* circulated in the Chinese public?
And if done deliberately is an option within “lab leak”, doesn’t that throw off the odds by several OOMs, given that for someone deliberately wanting the virus out there, but wanting to cover their tracks a bit, “releasing” it in a wet market that had all those animals would be a fairly high probability thing to do (in addition to being a very good way to ensure spread)?
Is deliberately put out into the world its own separate possibility, or is it incorporated into “lab leak”? And if the latter, what is faulty about my reasoning that it throws off the supposed unlikelihood of the outbreaks being centered in the wet market by many OOMs?
I loved this whole thing, thank you for writing it and I'm glad you enjoyed it!
I have no dog in this hunt and don't feel like any important policy-making we have to do hinges on this outcome. We know we need to keep labs safe and we know we need to deal with zoonotic risks. And I still found it riveting to read, as well as all the comments here. I think this lands in the genre of competency porn and I'm here for it.
A meta-observation: I started today 50-50 on this question. Right after reading it, I was briefly at 90-10 zoonotic. An hour or two later, I'd say it slipped to 75-25, and now I'm landing maybe at 60-40. That's an interesting experience just to watch over the course of a day. I don't know, but I think if I read it all over again, it might goose me back up to 80-20, temporarily.
When I ask myself why looking at the debate as presented swings me a lot but then that regresses over hours, I would summarize that as "mistrust". Mistrust of the institutions that produce information, mistrust of the arguments presented, mistrust of the biasing potential of the debate format. That's my emotional/psychological bias and so the further I get from direct contact with the elements of the debate (which I found persuasive), then that default bias starts to take over more.
While my intuition listens along to the content of the debate, it also has some deeply-held opinions about the trustworthiness (or lack thereof) of experts. I have my hobbyhorse example of this around guidance to parents for what position to put their infants down to sleep -- which has changed three times since my adulthood. It's not that anyone was misleading anyone, but that things that look absolutely true to science in one era turn out to be absolutely wrong in the next era and that's how science proceeds. We don't know what we don't know, and that makes all my reactions to these kinds of competing stories pretty uncertain. And it also makes me marvel at other people's certainty.
I don't know, but I think this might have some relevance for questions people ask about how to persuade people about the importance of large scale risks based on a mish-mash of reasoning and data. Where and how mistrust shows up is a huge issue.
> Fourth, for the first time it made me see the coronavirus as one of God’s biggest and funniest jokes. Think about it. Either a zoonotic virus crossed over to humans fifteen miles from the biggest coronavirus laboratory in the Eastern Hemisphere. Or a lab leak virus first rose to public attention right near a raccoon-dog stall in a wet market. Either way is one of the century’s biggest coincidences, designed by some cosmic joker who wanted to keep the debate stayed acrimonious for years to come.
Scott messed up. The lab was built near wet markets because wet markets were a threat, and GOF research probably was conducted there because leaks could be blamed on wet markets.
The new notes USRTK obtained through FOIA in January 2024 aren't trivial. I had been pretty much on the fence about the lab leak hypothesis, but the new evidence is so overwhelmingly damning it literally made me nauseous reading it. The notes explicitly propose generating a consensus sequence with an inserted furin site. They propose using six BsmBI sites to assemble the infectious recombinant - confirming a 2023 hypotheses that the spacing of the six BsmBI sites in SARS-CoV-2 had a suspicious spacing. The notes proposed passaging the recombinant virus in human cell lines under BSL2 conditions.
It's a mistake to casually dismiss new smoking guns, all caps or otherwise.
"Why did the older strain start spreading later? Probably the virus crossed from bats into raccoon-dogs on some raccoon-dog farm out in the country. It spread in the raccoon-dogs for a while, racking up mutations, including the (less mutated) Lineage A strain and the (slightly more mutated) Lineage B strain. Then several raccoon-dogs were taken to Wuhan for sale, including one with Lineage A and another with Lineage B. The one with Lineage B passed its virus to humans earlier. Then 3-4 days later, the Lineage A one passed its virus to humans."
So apparently the zoonosis story requires *two separate* zoonotic transmissions, from the same location and within the same week, but from separate viral lineages? Am I understanding this right? And if so, then can someone please explain why both sides are just passing over this claim as a minor aside rather than a devastating hole in the theory?
I am not an expert in the field, at all, but this intuitively strikes me as *really* implausible.
Side note about the use of "conspiracy theory" in this context. It's got these two meanings, one being "nutty, unfounded belief" and another being "a theory about how people coordinated actions secretly behind the scenes." There's a Venn diagram where flat earthers believe an endless stream of government officials are conspiring to keep the truth from us.
It doesn't seem accurate to me to call the lab leak idea conspiracy theorizing in the "nutty, unfounded belief" sense. Smart people in very recent years, including in this debate, disagree. When you accuse the other side in that situation of "conspiracy theorizing", you're just engaging in ad hominem attacks which add no useful information.
Some lab leak people do believe that there was behind-the-scenes conspiring that happened, but so do some zoonotic supporters. There are more people on the lab leak side who do seem to be more of the nutty, unfounded type of believers. But it's a distraction to look at them, because the evidence in favor of lab leak is not like the evidence in favor of the earth being flat, etc.
Sometimes too the people we call conspiracy theorists are just people who have been slower to catch up to the preponderance of evidence, given that science is a process that unfolds over years. I don't think it's helpful to call people in this category conspiracy theorists.
Strong arguments were made on both sides of this. Peter's arguments look stronger. But as a person who doesn't have their hands on all the raw data, I wouldn't say that he "debunked" the other side. He just presented some other information that was more persuasive.
Anyway, I guess this is a wish for not calling people "conspiracy theorists" as it relates to things scientists themselves are still debating.
The whole Session 2 issue seems irrelevant to me. The simplest lab leak scenario is a virus being collected in the wild, brought to the lab, then escaping in guano on a tech's scrubs or something similar. No need for genetic engineering of the virus at all.
The argument seems to have been "natural virus in wet market" versus "engineered virus from lab" while neglecting other scenarios.
"As mentioned earlier, the DEFUSE grant was rejected."
I do not know how many people here are familiar with how the federal grant system (whether NIH or DoD) for biomedicine works, but if one claims that the only way experiments get done is if 1) they are proposed in a grant and 2) that grant is funded, they are either unfamiliar with how the system works (like Peter) or else engaging in intentional misdirection (like the many virologists who have made this claim and certainly know better). As a general rule, the proposed experiments in a biomedical grant fall (often roughly equally) into 4 categories: 1) work you have already done but not published and inot included as preliminary data (you have to include just the right amount of data to justify the proposed experiments/money) ; 2) work you never intend to actually do but which looks good on the application; 3) work you do during the 1-2 years the grant is under review but no decision has been rendered; 4) work you actually will end up doing during the funding period.
That the DEFUSE grant itself was not funded is drastically less important (frankly not important at all) from the perspective of a Bayesian trying to figure out the origin than the fact they proposed making the precise modification that is so striking about Covid and has not been seen in any close relatives. David Baltimore gets this, Nick Patterson gets this, and I think anyone with knowledge of both how these grants work as well as the molecular biology gets this.
Saar raised the issue of ascertainment bias, but undersold it. There's no question imo: the case search *was* heavily focused on HSM, and the HSM sample search *was* heavily focused on the southwest corner. The Chinese authorities who did the search have been clear on this (Bahry, 2023, Table 1 and Fig. 1).
E.g. we now know that the raccoon dog stall was sampled 10 times, with 5/10 positive; while the pool of blood outside a beef stall in the east wing was only sampled once, with 1/1 positive (Liu et al., 2023; cf. Bahry, 2023, Fig. 1). That, not the racoon dog stall, was the highest positivity ratio. Peter's heatmap was based on a paper by some Western scientists who had no idea how many times each stall had been sampled, and had instead speculated that every sampled stall had been sampled equally (Worobey et al., 2022).* After Jesse Bloom also showed there's no positive association between raccoon dog and SARS-CoV-2 genetic material, nothing implicates them, not even in a God's-joke way (Bloom, 2023).
China CDC director George Gao has also emphasized, when asked by the host of a BBC podcast, that this overwhelming ascertainment bias means you can't assume genuine case clustering around HSM (https://www.bbc.co.uk/sounds/play/m001ng7c, 24:00–25:12).
Bloom, JD. (2023). Association between SARS-CoV-2 and metagenomic content of samples from the Huanan Seafood Market. Virus Evolution 9: vead050. https://doi.org/10.1093/ve/vead050
Worobey, M. et al. (2022). The Huanan Seafood Wholesale Market in Wuhan was the early epicenter of the COVID-19 pandemic. Science 377: 951–959. https://doi.org/10.1126/science.abp8715
*This zoonosis-theory paper is one of many by a prominent coauthor network centered on "Proximal origin" authors Andersen, Holmes, Rambaut, and Garry.
Question for Peter: don't you think you're being a little credulous with respect to the early case map, given an obvious and known effort from WHO/China to support anything but LL?
Separately--great job. To go into this with nothing but your own research and bet a good portion of your net worth against some rich guy, and win, is something from a movie. You should be proud for sure.
Well, all of this just confirmed my priors: most debate is utterly worthless. They can talk about likelihoods and logical fallacies all they want, but the problem is that no one has any hard evidence. Peter's obviously the smartest one in the room here, but just being intelligent is of no use if you don't have anything to work with. Sure, you can take a bunch of circumstantial evidence and then claim that they make your theory more likely by pulling numbers out of your ass, but none of that gets you any closer to verifying the actual truth. And at this point, actually verifying the cause of COVID is likely impossible-- any hard evidence would have been burned by the CCP, since even they don't have anything to gain by knowing the truth.
...So what's even the point of all of this? As others have pointed out, the actual dangers of wet markets or GoF research is completely independent of whether or not COVID in particular was caused by either of them. One event doesn't change the reality of the systems at play. We already agree on the information that matters: that the situation was potentially preventable by the CCP, and that they failed to do so. But even that wasn't enough for countries to stop doing buisness with them, so how would figuring out the origin of COVID change anything? This entire discussion is utterly pointless.
This is a highly interesting topic. Unfortunately I don't have the energy tonight to get deep into all the great points brought up in the post. But I want to share a few scattered thoughts in babble mode:
(1) Taking a 5% chance of dying on a climb is pure insanity. Maaaybe taking a 5% *lifetime* risk is worth it if you really love climbing. Hardly an endorsement for risking $100k at 4%.
(2) It's a terrible idea to offer a standing bet on your true beliefs. You're literally doing adverse selection and end up with a sneaky rules lawyer as your debate "partner".
(3) It seems that Peter is just a good debater, of the "debate club" variety. Laser focused on winning the debate, rather than finding truth. I was repulsed by the dirty arguments made, for example the quibbling about whether to call the first outbreak a "superspreader event" or not. Regardless of the completely irrelevant doubling times Peter brings up multiple times, a place like the Wuhan market clearly is a mechanism for turning a handful of cases into 80k cases. I've only skimmed the *summary* of the debate, so please point me to where Peter tackles the real underlying issue head on.
(4) Re: orders of magnitude. There is no limit to how badly you can fuck up a bayesian estimate. For example, Peter's 1 in 10'000 probability of the first infected person working at the wet market. That's just a uniform prior! It's so, so bad. The easiest way to debunk this is to take these odds, and start listing conclusions that would need to follow if they were actually correct: "viruses are equally likely to spread to everyone regardless of their profession and their position in the social graph"... um no, everyone with an inch of common sense (or kids) knows that viruses spread where people gather: schools, workplaces, and yes, markets. The vendor at that market didn't need to be the first, merely among the first *few*, and then keep coming to work while contagious.
(4a) It strikes me this might be a valuable method to flesh out. Let's call it the *reverse inference test". I don't know why, I just feel like it latches onto my intuitions more cleanly than trying to refute a claim in the same direction it's made.
(5) Probability cascades are bullshit. To me, this is the classic example of "science as attire"; you pretend to be scientific by multiplying numbers together, but what you're really doing is spinning a narrative. It's extremely hard to come up with an exhaustive enumeration of posibilities. MECE, as it's sometimes called (mutually exclusive, collectively encompassing). Or for the mathy people, an equivalence relation on the possible states of reality. If you doubt this, go take an advanced algorithms class and report back. I'm not joking... repeated encounters with impartial yet implacable entities (compilers, test suites, theorem provers, etc) will drill this into you like nothing else. Peter's argument style is that of a college freshman, and any competent math mentor would beat it out of him before it rots his brain. To the extent Peter is aware of this, it just reinforces (3).
(5a) to clarify the above point, the typical mistakes made is multiplying together probabilities that aren't independent, and to implicitly assume that the only way to reach state X is the specific path towards X you've laid out. These can make a 50% event seem like a 1% event, or the other way around.
(6) Explicit bayesian calculations in the absence of a *formal model* are a horrible idea. It just takes confusion and multiplies it by orders of magnitude. What I've learned in my time as a scientist is that the relationship between the real world and the model you're working on, and how to transport assumptions and conclusions back and forth between them, is one of the hardest things to get right. And we don't really have a systematic way of teaching this skill.
(7) If you wanted to blackpill me on the applicability of bayesian calculations to everyday life, you've succeeded. In particular, I see no reason to update my beliefs regarding the lab leak hypothesis in response to this debate. I'm still 50%+ in favor.
Wow, that was a lot more than I thought. Thanks for reading, and I hope it was helpful, even in this low-effort format.
My brain skipped a beat when I read the summary that Peter dismissed one of the later market spreads as "that didn't come from a human source, there were COVID particles on the fish". That feels eerily lab leak mechanics adjacent and I found myself slightly confused why he thought that was likely but COVID from the lab coming to the wet market and then spreading from there unlikely. Probably if I watched the videos, I would be less confused about this, and I'm not trying to make an argument, just registering my confusion.
Generally, I have no strong position on this question at all and a low prior on that the things I think about it are true. I thought this article was a fantastic deep-dive! I realise I could get even more information if I watched the videos, but I'm not really a video person. Thank you so much for this. :)
Regarding the 1 in 10000 chance of being discovered in a wet market if lab leak:
That is flawed. You might as well assign 1 in 365 that it was discovered December 30th. You can tell it is flawed because if it had been discovered in a school, Saar would assign 1/10000. Or in a restaurant, 1/10000. Or a bakery, 1/10000. He gets a free 1/10000 wherever because any public place can be differentiated by at least 1/10000. “Only 1/10000 Wuhans work at Wuhan IHOPs!”
You nod at this briefly in the conclusion, but what I find fascinating about this debate is that the what truly happened doesn't seem to matter at all! If an oracle arrived and told us the source, I don't think it would (or should) change our planning at all. The investigation of COVID has revealed that there are huge risks from *both* lab leaks and zoonosis that we need to invest significant effort in reducing, and which one actually happened in this case doesn't change that.
Maybe I'm missing something, but it seems like Peter's argument hinges on two separate cases of zoonosis within days of each other in the exact same location and the same mutation? Or maybe I misunderstand that point.
>The six estimates span twenty-three orders of magnitude. Even if we remove Peter (who’s kind of trolling), the remaining estimates span a range of ~7 OOMs. And even if we remove Saar (limiting the analysis to neutral non-participants), we’re still left with a factor-of-50 difference.
In a way, the scatter in estimated here is even more disturbing, because AI risk unavoidably involves estimates of future events, many with only weak analogies to known past events. _This_ estimate is all about events which _do_ have strong analogs to known past events.
I don't know what to suggest. One large class of worries is indeed treating some probabilities as uncorrelated when one could indeed expect correlations. E.g. as you said, researchers modifying viruses in ways intended to be less researcher-like and more nature-like in order to anticipate natural changes.
One other thing that generally bothers me is the blanks in a lot of the probability calculations where one side estimated an effect and the other side just didn't discuss it. Maybe it would have been helpful to have a final pass where all sides gave estimates for _all_ of the factors?
I truly hate to be a meta-commentator refusing to engage with the meat and bones of the debate, but I'm still puzzled by why this matters enough for the debate to be worth at least 100000 dollars and a few dozen hours of a few intelligent people's time (probably a lot more time and money, actually, given the extent to which all of this was discussed in other sources, of varying levels of sophistication, that I don't follow).
Would it help anyone in particular, even people who lost loved ones or whose businesses did not survive the economic consequences of the pandemic, to know whether the virus was zoonotical or originated from a WIV leak?
I doubt there would be meaningful political or economic consequences either way. I'm moderately sure virological research is still being funded about the same way it did before the pandemic, I'm a bit more sure wet markets are still as unsanitary as they were before the pandemic, I'm quite sure trade/political relations/economic dependence of the world on Chinese economy and industry is about the same as it was before the pandemic, and I'm very sure human beings still die the same way they did before (and during) the pandemic.
Great write up, and incredibly intellectually stimulating. One thing that struck me is how much harder it is to understand the past than to predict the future! If Rootclaim is so good, it should prove itself on the prediction markets, instead of relying on judges. This also solves the problem of financial imbalance, challenging people to high stakes debates is like bullying someone at the poker table when you have a pile of chips.
Importantly, Miller incorrectly claimed the N501Y mutation would result from passage in hACE2 mice (mixed them up with BALB/c mice). The major papers Miller relied on have been seriously challenged since the debate. See Stoyan and Chiu (2024), Weissman (2024), Bloom (2023) and Lv et al (2024). Overall the circumstantial evidence makes lab v plausible:
1. Chinese researchers Botao & Lei Xiao observed lab origin was likely given the nearest known relatives to SARS-CoV-2 were far from Wuhan. Wuhan Institute of Virology (WIV) sampled SARS-related bat coronaviruses where the nearest relatives are found in Yunnan, Laos and Vietnam ~1500km away. They refuse to share their records.
3.Molecular data: Only sarbecovirus with a furin cleavage site. Well adapted to human ACE2 cells. Low genetic diversity indicating a lack of prior circulation (Berche 2023). The CGG-CGG arginine codon usage is particularly unusual but used in synthetic biology.
4. DEFUSE full proposal: virus 20% different from SARS1, consensus seq assembled with 6 segments, without disrupting coding seq, BsmBI order, FCS. SARS2: 20% different than SARS1, 6 evenly spaced fragments w BsmBI and BsaI restriction sites, FCS.
Jesse Bloom, Jack Nunberg, Robert Townley, Alexandre Hassanin have observed this workflow could have lead to SARS-CoV-2. Work often begins before funding sought or goes ahead anyway.
5. Market cases were all lineage B. Lv et al (2024) indicates there was a single point of emergence and A came before B. So market cases not the primary cases. See also Bloom (2021), Kumar et al (2022). Peter Ben Embarek said there were likely already thousands of cases in Wuhan in December 2019.https://t.co/50kFV9zSb6
8. Lineage A and B only two mutations apart. François Ballox, Bloom and Virginie Courtier-Orgogozo note this is unlikely to reflect two separate animal spillovers as opposed to incomplete case ascertainment of human to human transmission (Bloom 2021).
9. Sampling bias. George Gao, Chinese CDC head at the time, acknowledged to the BBC stating they may have focused too much on and around the market and missed cases on the other side of the city. David Bahry outlines the documented bias. Michael Weissman has shown this mathematically.
Miller incorrectly claimed the N501Y mutation would result from passage in hACE2 mice (he mixed them up with BALB/c mice). Since the debate the key papers that Miller relied on have been badly undermined:
2. Lv et al (2024) found new intermediate genomes so the multiple spillover theory is unlikely (it was anyway given lineage A and B are only two mutations apart). Single point of emergence is more likely with lineage A coming first. So market cases are not the primary cases (all lineage B). Their findings are consistent with Caraballo-Ortiz (2022), Bloom (2021).
3. Jesse Bloom showed again the market samples don't support market origin. t.co/rorquFs1wm
4. Michael Weissman shows mathematically ascertainment bias in the early case data the judges relied on (George Gao, Chinese CDC head at the time, acknowledged this to the BBC last year - they focused too much on and around the market and may have missed cases on the other side of the city).
5. Account that identified errors in Pekar et. al. leading to an erratum last year has found another significant error. Single spillover looks more likely. t.co/GAPihZu51P
> I have a weird urge to visit Wuhan as a tourist, see the Wuhan Institute of Virology, stroll through the Huanan Central Seafood Market (unfortunately closed), maybe eat a raccoon-dog.
Scott N Alexander N plans N to N eat N Raphtalia N CONFIRMED
Thank you for an incredible write-up and analysis.
I am actually shocked at how poorly Saar fares here. The vast majority of his evidence is basically "if things were different in a way we can't know, then our conclusions would also be different, therefore lab leak." E.g. "if COVID spread rate was lower in the beginning", "if an infected WIV worker spread to the wet market then isolated", and so on. That's it, really? I was expecting so much more actual evidence. And then what data points they do provide (e.g. "Mr Chen", 90 cases, intermediates, etc) have apparently valid reasons why they're not evidence. I started out 60/40 zoonotic, but based on this I feel I have no choice but to update to 99/1.
I feel the real problem with this setup/framework is that a specific position is taken, with overly-precise numbers, and the goal of the debate is to simply justify that final percentage. You see this with Saar's "but the judges just did the math wrong" sour grapes complaint.
In the IT and infosec world, we have the FAIR method--a similar method for building quantified models of risks. The real value of these isn't that you convinced yourself you have only a 1-in-10000 chance of being compromised by a state actor in the next 10 years... it's the shared understanding of the risks your business thinks are important, how you think those risks are correlated, how you think they're evolving over time, etc. That enables your organization to act from shared goals, and not get bogged down in interminable debates day-to-day. The final numbers are fine, especially if you use confidence intervals, but you know it's only a snapshot in time using your best data and estimates as of the last time you underwent the exercise; by the time you update in a year/quarter, you will have more data and different estimates. Sometimes, risks that weren't even on your model last cycle, are now the top priority because of what's happening in the world (or because you gained new information that you missed previously). It's a fact of life.
With that in mind, I think Saar fares better if his arguments are framed as reducing absolute confidence in zootonic origin, and not the reason why LL is the only possible explanation (which it obviously isn't, even from Saar's own arguments). So yeah, I was only joking about updating to 99% in favor of Z--I think there's enough doubt thrown in from Saar's evidence alone that it could not possibly be more than 80%. But Peter/Z still wins this version of the debate, and Saar's over-confidence in "no way it's anything other than LL, here's the math to prove it, and if you disagree then you're doing the math wrong" does border on pseudoscience here.
To that end, I think Scott's final takeaways are the most important. If we have any shared goals anymore as a society, then we should devote resources to better preparing for pandemics from both origins, and in also improving government/organizational transparency so it's easier next time to understand what went wrong. As it is, either a natural disaster can be ignored because "teh government conspiracy," or else somebody got away with murder; either way, the debates that focus solely on convincing everybody that one's favored position is the only explanation means we're not more prepared for future disasters than we were with this one, we're just more divided and easier to manipulate.
I found this debate to be very valuable. I started with 70:30 odds on lab leak's side (I didn't do any extensive research on COVID origins) but after this debate that shifted to 5:95 (i.e., favoring zoonosis). I wish there are more such debates on contentious topics.
The Rootclaim model seems pretty sensitive to modest differences in probabilities; I'm guessing a factor of having too many variables so small differences across all of them add up to a big effect? I tried to average the six guesses giving Saar the least weight and add some of my own intuition and got (I think) a modest pro-lab leak outcome. https://docs.google.com/spreadsheets/d/14OkrvJ21qCqQcXINQnMO4-a1Xi-npidF/edit#gid=2057332411
> About half of cancer patients lose their hair, and Putin hasn’t, so we’ll divide by two.
This leapt out as methodologically unsound. How do you know whether Putin's lost his hair? Here are some possible scenarios:
1. Putin has never had cancer or anything cancer-like. His hair is fine.
2. Putin has had chemotherapy. He wears, or has worn, prosthetic hair to hide this fact.
Note that these two scenarios are perceptually identical. You have no way to tell one from the other.
If the question you're evaluating is "Is Putin covering up a case of cancer?", his visible hair provides no information at all, because all scenarios predict visible hair. In the "no" case, the hair is irrelevant, and in the "yes" case, the hair is deceptive.
It's kind of like making an adjustment to your evaluation of "Is this a coral snake, or is it a king snake?" for the observation that the snake in question has red and yellow stripes. That observation was already part of the premise of the question; you're double counting (among other weird mistakes) if you adjust for it again.
> Another of Saar’s concerns with the verdict was that Peter was an extraordinary debater, to the point where it could have overwhelmed the signal from the evidence.
I've mentioned this before in the ACX comments, but this is such a great lead-in that I'll say again that this is why I didn't make a habit of reading Overcoming Bias. Eliezer Yudkowsky used that blog to make a lot of excellent points in an entertaining and well-written fashion.
And I got the sense, reading his work, that sounding persuasive was a bigger part of his strategy than being right, meaning that reading him was dangerous. So I enjoyed a number of pieces, but I didn't follow him or seek out his work.
My suspicion here is that using estimated probabilities instead of probability distributions leads to the same issue as for the Fermi paradox, in that your result is extremely sensitive to your assumptions and you end up with extreme outcomes on either side like we have here: 10^-25 and 534 are both clearly wrong.
My overwhelming feeling while reading this was a sense that a lot of the important evidence is itself subject to long debates and explanations that are being glossed over. Is a certain length of protein sequence likely or unlikely to be done by a mutation in the wild? What is the right way to construct priors about wet market and virus lab locations? Everyone was just stating certain things confidently, while providing detailed documentation for other things.
In the end, “evidence for my claims” seems to have a fractal quality sometimes. The closer you look, the more evidence you need to back up your use of evidence. It’s turtles all the way down.
While I'm terribly sympathetic to most of Wilf's points, one thing I absolutely agree with him on is the superiority of written to real-time/spoken debate, because the things it rewards correlate much more strongly with being right.
It seems to me that Saar's evidence-weighing analysis is basically "outside view" with extra steps.
As I see it, Saar's analysis hinges on the point that the zoonosis hypothesis contains a coincidence that is very unlikely to be explained by some out-of-model error (which I agree with), whereas the lab leak hypothesis only contains a coincidence that could very plausibly be an out-of-model error (I agree with this as well). This sets up a barrier that is almost impossible for any amount of mere digging into the details to overcome, as that details-digging is all itself quite vulnerable to out-of-model errors. And so the end result is an outside-view judgement that no amount of inside-view analysis can outweigh.
This surely seems like an error to me, though those with different positions on the outside-versus-inside-view debate might disagree.
I genuinely do not understand why there is not more talk about the third option: that one of the many bat cave spelunkers from Wuhan caught COVID-19 and brought it back to Wuhan.
The WIV and the market were fairly far apart from each other, as noted in the debate. The Wuhan CDC is *right next to the market*. And while it doesn't seem that the Wuhan CDC was conducting gain of function research, their employees were fanning out all over China going into bat caves.
"In the video, the researchers scale the cavern wall, their headlamps ghostly blue.
“If our skin is exposed, it can easily come in contact with bat excrement and contaminated matter, which means this is quite risky,” says Tian Junhua ["associate chief technician in the Wuhan CDC’s pest-control department, but he has a reputation as a swaggering adventurer in his work with bats and insects"], one of the bat hunters.
...
The video was released by national science authorities and Chinese state broadcaster CCTV on Dec. 10, 2019, and circulated on social media. It’s a high-quality production, designed to promote China’s world-leading viral research.
...
Tian and his team from the Wuhan Center for Disease Control and Prevention are filmed catching horseshoe and pipistrelle bats and collecting samples of guano, in search of new bat-borne diseases and the basis of new vaccines. Tian talks about the need for caution. “It is while discovering new viruses that we are most at risk of infection,” he says, though he is shown handling sample vials without wearing full protective gear.
...
In 2017, Tian told the state-run Wuhan Evening News he once forgot personal protective equipment and was splattered with bat urine, leading him to quarantine at home for two weeks. On multiple occasions, bat blood squirted onto his skin while he was trying to grasp the animals with a clamp, he told the paper."
So the questions would be - exactly when did the Wuhan CDC move to be close by the market? The move finished on December 2, 2019 - how long was the move and how was it conducted? Does the science require an intermediate animal or could it have jumped directly from bats to humans with the cave exploration?
I am now scared that the next pandemic will be started by debaters on either side doing gain of function research to shore up the evidence for auxiliary points in their debate.
There's so much stuff on this page that Chrome is having difficulty rendering it even on a high-end desktop. Consider closing comments and creating a separate page for that.
At the outset of the pandemic, I was at about 15% lab leak, 85% natural origin, and thought it unimportant because I didn't think we'd ever have enough evidence to do better than guess. By the start of this year, after way more than 15 hours of studying way more evidence than I imagined we would ever have, I was at 80% lab leak, 20% natural origin, After this, maybe I should update to 70% lab leak, 30% natural origin. But I'm going to indulge in a bit of emotion and make that 25% lab leak, 100% annoyed.
Annoyed in that it seems like the rationalist and rationalist-adjacent community, parts of which at least I value and respect, has decided that on this one issue we're going to flip a $100K coin to decide what to believe, and I suspect anyone who dissents will at best be quietly dismissed as an ignoramus by much of the community. I will grant that by paying $100K, we got a coin that is *slightly* biased towards the truth,
Seriously, one-on-one personal realtime debate is a *lousy* way to find the truth. There's a reason we don't have debates as a central feature of e.g. scientific conferences. Throwing in high-value bets doesn't improve things. The process selects for the most skilled and charismatic debater, and in some cases for the favor of biased judges, with the truth mostly just serving as a tiebreaker.
In this case, there was no tie to break. Miller, by all accounts including this one, is an exceptionally capable debater. Wilf, while clearly skilled as an entrepreneur and possibly as an analyst, does not seem to be very good at in-person debate. From this very thorough account, Wilf completely failed to raise some of the very strong points in favor of the lab leak hypothesis, failed to call out Miller on some of his indefensibly weak points, and agreed to a structure where he'd open by spending the first third of the debate on the very weak assertion, "maybe it didn't come from the Wet Market" even though that is in no way necessary to the lab leak case. Or maybe he did raise the strong points and call out Miller's weak points, but in a way that neither Scott nor the judges noticed.
As for Scott shifting is odds from 50-50 to 90-10 on the basis of this debate, I think that requires p<0.20 for "Wilf isn't very good at this and left the winning arguments at home", which I don't think is defensible. You'd need a statistically significant sample of debates to be confident you have seen both sides well-argued.
I will continue to value and respect the rationalist and rationalist-adjacent community, particularly Scott's corner. But it has a clear blind spot in its enthusiasm for clever and novel truth-seeking mechanisms (bets, prediction markets, debates, etc) on the basis of their cleverness and novelty without adequate regard for their in many cases obvious flaws.
If it turns out it was natural origin — as I am more likely to now believe — it’s sad it will be that much harder to end gain of function research. Even if the lab did not leak the pandemic, the risk that the lab would leak a pandemic eventually was still too damn high. Hopefully this event will nonetheless lead people to take the risks more seriously.
I’ve spent some time in virologist subreddits since covid, and I find their attitude towards these risks to be terrifying. They remain very blasé about it. It appears to be deep in their culture, similar to how civil engineers are culturally committed to designing for cars, and architects are culturally against ornamentation.
The science has to decide whether it’s a lab created virus or not. However created in a lab is not the same as lab leak. In fact your prior for this being a leak should be close to, or at, 0% since it’s the only time a major pandemic has happened in that fashion.
When you look at the distance between the lab and the market and the fact that the origin is clearly the market if lab created pathogen is proven true then the most likely explanation is an external attack.
> So for example, suppose there are fifty things about a virus. You should expect at least one of those to have a one-in-fifty coincidence by pure chance.
Minor math error here I think: You should expect *exactly* one, not at least one. It's equally likely to be more than one or less than one.
Thank you to Saar for running this in good faith, and thank you to Scott for a great write up. One of the key things that stood out to me while reading the analysis was that there are a few relatively simple factual items that were disagreed on that might be relatively easy to chase down if we weren't worried about governments trying to obfuscate things. There's also a few simple pieces of information that could shift my thinking a bit too, like where do the lab workers live? I wouldn't expect a lab leak outbreak to be centered around the lab, but around one of its employees homes.
I noticed in myself a high risk of falling victim to a classic 'david and goliath' story too, which is potentially a very bad bias. Almost everyone reading that first section is going to be rooting for Peter, and enjoy it everytime he seems to win a point.
I must admit that I don't care nearly as much about whether COVID was zoonotic or not, as I do about the government paying social media sites to enforce a particular orthodoxy about the question.
Some of this sounds like an “Encyclopedia Brown” level of argument. E.g. the idea that because the University of North Carolina said they’d do furin site insertion in a grant application from years before the pandemic, therefore the WIV cannot have done any insertions of their own.
Research institution capabilities, and researcher ambitions, and the real world generally, are a lot more complicated than that reasoning implies (I say this despite putting little weight personally on COVID being engineered).
Likewise with the idea that the WIV would have fully released the details of all viruses they have sequenced.
I haven’t watched the debate though, so maybe it was better than Scott makes it sound.
It strikes me that whether you should debate theories you think are crazy depends on how popular they are. Consider:
1/10000 believe in theory A. You debate someone arguing A, and you win 99% of the audience (considering what humans are like, this is super impressive). But now, 1/100 of the audience believes in the crazy. And *no-one* can convince 99.99%.
66% believe in theory B. Now when you debate someone arguing theory B, even if you only convince *half* the audience, things have improved!
This, of course, assumes that the people believing what you think is crazy can be convinced in the first place. This might well be true about lab leaks or *possibly* even UFOs; it will *not* be true about creationism (evidence isn't even a factor in why someone is a creationist).
The JFK assassination is another of those cosmic jokes. Sensible people should not believe in conspiracy theories about it. But dear Lord, was Lee Harvey Oswald a weirdo, the rare person who had been involved with *all* the kinds of people who _might_ be suspected of conspiracies; and murdered before he had a chance to testify; and with an investigation that couldn't have seemed more suspicious and cover-up:y if it tried.
Really fantastic read. I already leaned towards zoonosis but this removed most of the remaining doubt. It so valuable to see one side put up their strongest arguments and then see if the other side can knock them down, which in this case they did pretty dramatically.
“… And so on and so forth, until we end up with the final calculation: 86% chance Putin doesn’t have cancer, too bad.”
What Rootclaim is doing is interesting. But should not such hard-core Bayesianism also include sensitivity analyses? That is, investigate how robust the conclusion is (here: “86 percent chance”), at least for minor-to-moderate changes in priors, age-related cancer statistics, and the like?
Using sensitivity tests in cost-benefit analyses is usually recommended. It appears that Bayesianism could benefit from such tests, for the same reason.
It would also quell the skepticism people like me automatically feel when encountering predictions spelled out in fine-grained percentages. (Plus, why stop at percentages? Why not add a few decimal places as well: 86,xy percent chance Putin doesn’t have cancer– that would make the prediction even more “accurate”….)
Do not get me wrong – I respect what Rootclaim is doing. But I would be (genuinely) interested in why Rootclaim does not add/construct some kind of confidence interval around their percentage-estimates. (And why Bayesians in general do not do this.)
This post caused me to strongly update against Bayesian methods. Maybe it's the impossibility of seeing accurate priors, or the tendency to engage in cherry-picked absurd probability estimates. Or maybe it's the qualitative difference between statistical probabilities prospectively estimated vs retrospectively measured. I no longer have confidence this method produces more signal than noise.
For example, what prior do you set for an outbreak in WSM? You might think that's low, but given multiplicity you're not really looking at a prior probability anymore, are you? Both a spillover event and a lab leak might have come from any number of places. We'd tell some plausible-sounding story about why that place was unique, then set absurd priors as a fence around that story. But at heart it's still a post-hoc story, and all the prior-setting in the world will only serve to reinforce your story with more plausible-sounding stories. This seems to be what Saar is fighting against in making adjustments for out of model variance, but that seems like the wrong approach.
The better approach is probably abandoning Bayes - at least explicitly/strictly. I'm not yet convinced about the whole Bayesian enterprise, but it feels like it's on much more shaky ground for me today than it ever has since I first learned about it.
I personally agree it was not a lab leak but a pretty important was lost in the debate (or at least poorly factored in). Namely China was making hiding evidence. While this may impact priors…the bigger impact is that, if it was a lab leak we only know what information was released (which almost certainly would be anything that boosted their preferred narrative) and do nit have all the evidence that was presumably withheld (which would be all the evidence they could suppress that went against the preferred narrative).
It’s kind of like sample bias in that the evidence you use to make the assessment is systematically biased.
I unsure how you assess these probabilities because they are fundamentally unknowable.
"maybe you should lower your disbelief in each hypothesis to something more reasonable, like 99%. But now the chance that the sun rises tomorrow is 0.99^100, aka 36%. Seems bad."
All these hypotheses are very correlated; extreme probabilities of base events are not required, just extreme probabilities of conditionals.
'If all 99 other hypotheses are false, and I'm not making some silly maths mistake or whatever, then hypothesis 100 is 99.999999% false' is not necessarily overconfident even for a human.
I have done my own extensive research and believe the evidence is compelling that there was a lab leak. IMO, Peter has a real axe to grind and his strong personal conviction against the lab leak hypothesis has made him blind to common sense.
The odds that the virus evolved free of human-engineering are probably much less than in 1/10,000. Peter has no idea how to figure those odds and frankly no one else does either. The actions of the Chinese government in stonewalling are far more convincing to me that there was a lab leak. If the Chinese government didn't have something to hide why be so opaque???
For an adversarial setup (such as debates or US court cases) to converge on something close to the truth, the case for each side needs to be represented at a much more similar level than was done here. Even if an innocent man is accused of a crime (so he *knows* that the truth will set him free), it is still generally recommended that they hire a competent lawyer to represent their side, as opposed to defending themselves. This is probably not as important in countries with a less adversarial court system.
Calling Peter a better debater doesn't quite capture what happened here. I don't think there was a great difference in rhetorical skill, but there was an extreme difference in both the level of preparedness and the commitment to the cause. This case involves 50+ pieces of circumstantial evidence of various degrees of strength, some favoring lab leak and some favoring zoonosis. All the ones favoring lab leak were heavily contested, but at least to me it did not seem like the same was true for many of the ones favoring zoonosis. This was probably a mixture of a few things. Saar didn't seem to not know enough of the precise details to effectively push back against some of the bolder zoonosis arguments, such as how confident we should be that we actually have a good picture of the first cluster. In other words, he conceded far too much, allowing the zoonosis side to look much stronger than I suspect many people still believe it is. Secondly, I don't think Saar fully understood what his "role" in this debate was. Instead of being a soldier for his side, he mostly stuck with his 2-3 year old argument for what he thought the "truth" was. Peter, on the other hand, seemed eager to actually win the debate, so he made the maximally strong argument for his cause, not granting anything, as opposed to trying to converge on the "truth". So for every piece of evidence, he was able to put on the "maximally pro zoonosis" spin, perhaps resulting in his absurdly low final lab leak probability.
It is perhaps interesting to think about how this compares to adversarial collaborations, which are popular in our community. Using the language of Julia Galef, one can perhaps think of an adversarial collaboration as a "dance" between two scouts. Similarly, a debate can perhaps be thought of as a battle between two soldiers. The incentive is to win, not to find the truth. The rootclaim debate looked like a battle between a soldier and a scout, with the predictable result.
(Disclosure that I support in BiosafetyNow, but these are my views and don't necesarily reflect those of BN)
The Twitter origins debate, which Scott understandably shies away from (ref. his most recent tweets, and ofc I certainly don't blame him) represented the richest, most boisterous most tumultuous, most rancorous, most acrimonious form of the origins debate I've ever encountered online. The current ACX substack post & its comments don't reflect the Twitter debate's diversity of opinions, evidence, and individual credibilities IMO.
The key items I feel are insufficiently highlighted (even if they are mentoned at least once) from my scan of the comments section:
• the value of 'adverse inference' facing:
° PRC's gov
° the most outspoken virologists (+ 1 evolutionary biologist, + 1 disease ecologist, and yes not *all* virologists) especially when relevant materials turned up from their side only following subpoena
• No progenitor or 'sister' virus with an MRCA dated to shortly before outbreak has been found in animal populations despite extensive sampling.
• I don't feel any post-outbreak (and some near-pre-outbreak) data out of the PRC should be accepted on faith; rather it & all conclusions drawn therefrom should be placed in its own little suspect epistemic box. By contrast, any data in non-PRC hands pre-outbreak is 'gold'. The remaining quadrant, data & analysis produced in the ordinary course ex-PRC but post-outbreak, is tainted by the need to avoid 'rocking the geopolitical boat'. Often we can't reference such data that for example underlies post-outbreak intel findings, like that of the ODNI (atop DoE, FBI, CIA DIA...) so I can only give it the 'silver' label treatment.
And transcending all these quadrants, verified data that turned up ex-PRC from leaks, FOIAs, sequences reconstructed from SDAs, etc. is 'platinum'.
1. Peer reviewed papers since the debate that show ascertainment bias in early case data and intermediate genomes that undermine the multiple spillover theory indicating lineage A arose first so the market cases are not the primary cases (Weissman 2024, Lv et al (2024), Stoyan and Chiu (2024).
2. Miller's claims about the furin cleavage site are dubious:
Miller incorrectly claimed the N501Y mutation would result from passage in hACE2 mice (he mixed them up with BALB/c mice). WIV was performing in vivo experiments in transgenic (human ACE2 expressing) mice and civets in 2018 and 2019 in SARS-like CoVs.
I think both sides may have also overlooked the MERS furin cleavage site shares several structural and functional similarities. The sequence looks quite similar. In 2019 WIV researchers were involved in MERS research. Yusen Zhou who died in mysterious circumstances in May 2020 and was involved in an early Covid-19 vaccine was also involved in this research that involved manipulating the FCS. t.co/7zcSUPR60T
Broad Institute biologist Alina Chan also observes the S1/S2 FCS PRRA insertion in SARS-CoV-2 generates a Class IIS restriction enzyme site (BsaXI). This was used by WIV and Ralph Baric at UNC previously.
Dr Andreas Martin Lisewski discusses similarities with a MERS infectious clone described in 2017 here. The argument no one would engineer an FCS like this overlooks the MERS example.
I think it's wrong to characterize this as a tragedy for Saar and Rootclaim. Sure the contest didn't validate their methods -- but I think their purpose was bigger than just winning internet arguments. Their goal is basically furthering scientific knowledge. Every model and method is flawed, but some are still useful. I see this as a heroic story, and Saar's stubbornness doesn't negate his accomplishment. By gracefully hosting and funding this exercise, he's made us all better off and advanced our knowledge of important issues. And that is the most important thing.
So the Chinese want us to think that they caused a global pandemic?
Although not infallible, humans are actually pretty good at ascertaining the intentions of enemies and potential enemies, and that is why it is a universal human inclination. Being oblivious to the likely motivations and intentions of other people is a big evolutionary disadvantage.
> Newtonian mechanics wasn’t pseudoscience when Newton discovered it, but if someone argued for it today (against relativity), that would be pseudoscientific.
I disagree strongly with the ontological assumptions here. Newtonian mechanics and relativity are both great theories. Newtonian mechanics is superior for terrestrial, human-scale phenomena because it provides a simpler representation of the physical laws. Relativity is superior at speeds approaching c because the complexity is justified by the accuracy gained. To claim that Einstein > Newton for modeling the trajectory of a baseball would be the same error as saying that Newton > Einstein for modeling a black hole: both arguments would be domain-of-applicability errors.
I self-identify as a flat earther, because for 99% of daily life, the flat earth theory (viz., conceptualizing the Earth as a flat plane) is simpler and more useful than conceptualizing it as a globe. The error made by "actual" flat Earthers is not that their theory is bad - they have an excellent and very useful theory - it's that they misunderstand their theory's domain of applicability. Flat Earth theory gets you to the grocery store; global earth theory gets you to the moon.
I think most points here are good points to make, but I also think it's useful as a general caution against this type of exercise being used as an argument at all! So I'd obviously caution against anyone taking your response itself as a reasonable attempt at an estimate of the "correct" Bayes factors, because this is all very bad epistemic practice! Public explanations and arguments are social claims, and usually contain heavily filtered evidence (even if unconsciously). Don't do this in public.
That is, this type of informal Bayesian estimate is useful as part of a ritual for changing your own mind, when done carefully. That requires a significant degree of self-composure, a willingness to change one's mind, and a high degree of justified confidence n your own mastery of unbiased reasoning.
Here, though, it is presented as an argument, which is not how any of this should work. And in this case, it was written by someone who already had a strong view of what the outcome should be, repeated publicly frequently, which makes it doubly hard to accept the implicit necessary claim that it was performed starting from an unbiased point at face value! At the very least, we need strong evidence that it was not an exercise in motivated reasoning, that the bottom line wasn't written before the evaluation started - which statement is completely missing, though to be fair, it would be unbelievable if it had been stated.
Counterposition here- given the verified attempts of multiple authorities to shape the "narrative" around covid, plus the powerful incentives for them to do so, does this mean that the sum of evidence available to assess the situation is probably unreliable and unfit for the purpose of this debate? The added layer of information control typical of the Chinese government makes the situation even more opaque.
Maybe this isn't a debate which can be settled based on the available information. Maybe that is by design.
The real question for me is this- why is any of this important in the long run? The obvious answer is that it has some bearing on how the world might respond to future pandemics. If it is yet another zoonoses then all the efforts to date to prevent them didn't prevent this one and probably wont prevent the next one, so maybe we should attempt to apply democratic pressure to make governments do more. I'm not sure they can do much more either way.
If laboratory research is behind covid, then this is also not completely unprecedented (though the level of genetic engineering might be somewhat new). Again, the technology isnt something that can be easily regulated by governments. Nucleotide sequence synthesis is relatively centrally controlled by a few large companies with the technical capacity, but that is only one of many ways to generate genetically modified pathogens. Again, we could attempt to apply democratic pressure to democratic governments, but again that would likely have limited impact.
The only thing that really shines through with the whole covid affair was the government and health authority overreach, panic, incompetence and opportunism. Once again none of this is new. The corporate capture of the medical system has been progressing for decades. At best the whole covid affair is just another reminder that authorities do not often have the interests of the individual as their main priority, and even when they do their capacity to deliver functional outcomes is usually highly limited. All trends point to this situation worsening in the future.
Novel pathogens will likely always remain potential black swans, which means that most of the time authorities will over react to their emergence relative to the actual risk posed. Over time this will probably build up broad social resistance to government responses until a genuinely threatening pandemic emerges. The relative difference in perceived threat between pathogens that are novel and fast moving versus those that are familiar and slow (such as tuberculosis) is also worth contemplating. Our information ecosystem thrives on novelty. Nobody is going to buy a newspaper that reports the same steady rate of slow deaths from tuberculosis or malaria. No government minister is going to win the fanatical devotion of the masses by heroically fighting such unexciting killers.
>"Peter seems to have a photographic memory for every detail of every study he’s ever read. He has some kind of 3D model in his brain of Wuhan, the wet market, and how all of its ventilation ducts and drains interacted with each other."
Mostly I think it's just keeping up with the Standard Arguments™️, from papers and preprints by zoo crew (Holmes, Andersen, Garry, Rambaut, Worobey, Pekar, Rasmussen, Débarre, Crits-Christoph, Neil etc.—the "Proximal origin" authors and their collaborators). They're the group of Western virologists who ~yearly come out with a new paper finally proving market zoonosis for real this time (Andersen et al. 2020, Holmes et al. 2021, Worobey et al. 2022, Pekar et al. 2022, Crits-Christoph et al. 2023a, Crits-Christoph et al. 2023b, etc.) and get a ~yearly media circus about it. Maybe he also read some pop-sci books like Chan & Ridley's "Viral" for pro-lab (2021) and Quammen's "Breathless" for pro-zoo (2023). Probably also the Chinese HSM-sampling paper (Liu et al., 2023) and WHO-China joint study (2021), following zoo crew in how to interpret them.
E.g. Peter's heat map slide is just Fig. 1 of (Crits-Christoph et al., 2023a), which is an elaboration of Fig. 4 of (Worobey et al., 2022); and his "drains" argument is from Fig. 2C of (Crits-Christoph et al., 2023b).*
Bloom, JD. (2023). Association between SARS-CoV-2 and metagenomic content of samples from the Huanan Seafood Market. Virus Evolution 9: vead050. https://doi.org/10.1093/ve/vead050
Chan, A. and Ridley, M. (2021). Viral: The Search for the Origin of COVID-19. HarperCollins.
Crits-Christoph, A. et al. (2023a). Genetic evidence of susceptible wildlife in SARS-CoV-2 positive samples at the Huanan Wholesale Seafood Market, Wuhan: Analysis and interpretation of data released by the Chinese Center for Disease Control [preprint]. Zenodo. https://doi.org/10.5281/zenodo.7754299
Crits-Christoph, A. et al. (2023b). Genetic tracing of market wildlife and viruses at the epicenter of the COVID-19 pandemic [preprint]. bioRxiv. https://doi.org/10.1101/2023.09.13.557637
Pekar, J. et al. (2022). The molecular epidemiology of multiple zoonotic origins of SARS-CoV-2. Science 377: 960-966. https://doi.org/10.1126/science.abp8337
Quammen, D. (2023). Breathless: The Scientific Race to Defeat a Deadly Virus. Simon & Schuster.
Worobey, M. et al. (2022). The Huanan Seafood Wholesale Market in Wuhan was the early epicenter of the COVID-19 pandemic. Science 377: 951-959. https://doi.org/10.1126/science.abp8715
*The heat map is misleading: it was based on implausibly assuming all sampled stalls were sampled equally; in fact the wildlife-corner stalls were sampled far more heavily (Liu et al., 2023). For a critique of dismissing ascertainment bias, see my (Bahry, 2023). For a critique of Crits-Christoph et al. 2023a, see Bloom (2023). For a critique of their later, also-misleading heatmap (Crits-Christoph et al. 2023b Fig. 2a), see (https://www.biorxiv.org/content/10.1101/2023.09.13.557637v1#comment-6279754825).
I find the "drains" argument mildly interesting (Crits-Christoph et al., 2023b Fig. 2c): nonzero evidence, but just one real datapoint. (The two downstream of the racoon dog stall are *in general* downstream of a big chunk of the western wing and the entire western wing respectively, so those mean little.])
Scott- I can roughly translate my factors into the grouping you use in your table.
Net priors using your grouping)
me: 12 you: 16
combo of all HSM/lineage factors
me: 1 you 0.002
FCS-ish
me 25 you 25
reasons WIV wouldn't do it
me: in priors you: 0.17
cover up success
you 0.5 me: did it?
other factor restriction enzyme pattern
me 70 you NA
The RE pattern is new, and I think you'll find my argument at least reasonable.
The big disagreement is about Worobey/Pekar. If I were to argue for ZW, I'd stay the hell away from Pekar, whose errors are now verging on something worse than "wrong".
Worobey does correctly point out that there was a spreading event at HSM, but so many specifics (lineage, internal DNA-RNA correlations, Wuhan share of relevant species (if any) don't fit that ZW has to rely on the non-market versions, which actually were generally considered before hand to have more of the priors than a market version- including by DEFUSERs.
Lots of people in the comments seem to have additional evidence one way or another that they want to add in. Maybe we need a coronavirus version of TalkOrigins.
I just cannot get myself to the point where I take anything being stated about this as reliable. For 2-3 years, everything that was being said about COVID by "experts" was either on what Zvi calls Simulacra level 2 (the public health establishment; saying X, despite ~X, to make you believe X and act in a way they want you to) or on what he calls Simulacra level 3 (Twitter users saying something not because THEY believe it's true, or even that they expect YOU to believe it, but just to signal that they're good non-racist Science-lovers). So even while I find Peter's points as you've summarized them to be pretty reasonable, he hurts himself IMHO by making the characterization of lab leak as a "conspiracy theory", because accusing an opponent of that also happens to be great tribal signaling. And the data on which all of this is premised comes from sources like the WHO which is clearly willing to lie for a variety of reasons.
So despite finding Peter's arguments here pretty reasonable, I can't get over the hump. I also, were I as dishonest as the PH establishment, would be inclined to just say it was a lab leak to induce public fear of gain-of-function research to help eliminate it. I can live with that having been wrong for this one virus, but still wanting to ban such research because next time it may NOT be wrong, but since most people don't reason that way then zoonosis being true is quite a blow to efforts to end this dangerous stuff. So let's give public health a dose of its own medicine and keep a nice healthy percentage on lab leak regardless.
Even after this excellent post, I find Michael Weissman's v5.7 analysis from March (https://michaelweissman.substack.com/p/an-inconvenient-probability-v57) to be the most compelling information available online to date, and it contains significant evidence updates from his v5.6 post that you linked in the write-up from January. I've kept an open mind through the years about Covid origins and found the Rootclaim debate to be especially interesting and cause for shifting my probabilities, but I have not encountered anything that has so significantly updated my probabilities than Michael's latest v5.7 analysis does. I'm curious if people have read that analysis in full and what everyone's thoughts are.
You seem to initially be interested specifically in Michael's Bayesian probabilities to compare his results to your table (which I think he posted recently in the comments above). I understand that you likely have dedicated dozens of hours to this post already and reading through a massive updated analysis is likely not especially appealing, but just looking at the probabilities alone does a disservice to the immense amount of research, evidence, and analysis that is included in Michael's write-up. For what it's worth, I've read an incredible amount of high-effort research and debate on this topic throughout years of following this, and Michael's research/analysis is the first one that I find convincing. In addition to more convincing Bayesian logic, his research includes crucial compelling evidence/information that came out after the debate (or was available but dismissed or potentially misrepresented at the time of the debate). I am trying to be as fair to both sides as possible and I honestly can't see how someone can read Michael's v5.7 post in full and come away with a 90% probability of Zoonosis. In my mental model, the two most likely reasons are that either people haven't read Michael's 5.7 analysis due to low exposure of a small blog, or I am fundamentally misunderstanding something.
Either way, I want to know what people think. If Michael's post isn't as convincing as it seems to me, I'd like to know why so I can update. If it is convincing, I'd like others to know so they can update. @Scott, I historically have placed a lot of value on your opinions and I'm genuinely interested in what your thoughts are here.
Personally, I don't have a lot of emotional/ideological investment in the issue so wouldn't sit through the debate but it great to get a detailed enough write up of the main arguments and their problems. I guess I should be over it, but I remain amazed by the level and vehemence of the responses here and elsewhere. Lab leaks remain possible whether or not it happened here so there's an ongoing risk anyway. There are plenty of risks. In the long term, I'm actually more worried about the intentional release of an engineered virus than an accidental leak.
100k is about 0.05% of Saar's net worth. I assume it is a much much larger share of Peter's net worth. More to lose. More to gain.
The incentive for debate performance is massively skewed toward Peter in this case. Worth noting when analyzing the truth-seeking utility of this format.
Sorry if this is noted elsewhere, it is a big comment section.
This was such an interesting read. There are a lot of smart people here in the comments continuing the discussion, but I just wanted to chime in and say THANK YOU for writing something so compelling to read for somebody (me) who has no prior knowledge or real opinion on the subject(s). Legend ⭐
"Either a zoonotic virus crossed over to humans fifteen miles from the biggest coronavirus laboratory in the Eastern Hemisphere."
As they say, 'you couldn't make it up'. Or, 'truth is stranger than fiction'. It does *seem* very suspicious co-incidence that a new virus that would turn into a world-wide pandemic just happened to pop up out of nowhere on the doorstep (as it were) of a specific institution dedicated to doing this kind of research, but there was no connection. And yet we seem to be forced to accept that this is so.
Congratulations and gratitude for sitting through 15 hours of video to produce this post! And I did appreciate the Linear B/Lineage B joke.
What you say about raccoon dogs here is mistaken. Raccoon dogs are not a plausible intermediate host for sars-cov-2 on the basis of information that has been known since 2021. There are several considerations.
1. Xiao et al (2021) - https://www.nature.com/articles/s41598-021-91470-2%E2%80%8B%E2%80%8B%E2%80%8B , which includes a co-author of Worobey et al (2022), a leading zoonosis paper states in table 1 that the raccoon dogs were wild caught in Hubei, not farmed as you assert in the piece. This alone rules out raccoon dogs as plausible hosts for two independently sufficient reasons. Firstly, there is unanimity in the literature that the bat ancestral virus to SARS-CoV-2 is in southern Yunnan or South East Asia. Everyone agrees with this, including Shi Zhengli. If a species was wild caught in Hubei, then there would be no explanation of how it acquired the ancestral bat virus, given that Hubei is 1000 miles from southern Yunnan.
Secondly, a mystery of sars-cov-2 is how it acquired the furin cleavage site that makes it so transmissible. There are 850 known sars-like coronaviruses, and only one with a furin cleavage site. According to private messages exchanged by proponents of zoonosis, the furin cleavage site could not have been acquired in the market because the density of animals was too low (only 3-4 per cage). When avian influenza acquires a furin cleavage site that occurs on farms with thousands of chickens densely packed, i.e. not in the wild and not when there are a handful of animals in cages in a market. https://usrtk.org/covid-19-origins/visual-timeline-proximal-origin/
2. Wang et al (2022) https://academic.oup.com/ve/article/8/1/veac046/6601809 also confirms that the raccoon dogs were wild caught in Hubei. What's more, Wang et al (2022) tested 15 wild raccoon dogs of suppliers of Wuhan markets, including the Huanan market, in January 2020 and found them to be negative for SARS-CoV-2. On average, 38 raccoon dogs were sold across the four markets in Wuhan from 2017 to 2019. So, the 15 raccoon dogs likely comprised nearly the whole inventory of raccoon dogs that would have been supplied to the Huanan market at the time.
In short, the raccoon dogs supplied to the market were tested and were negative. It is very strange that the raccoon dog narrative has persisted for this long given this public information. Even the strongest proponents of the raccoon dog hypothesis have walked back their bold claims that raccoon dogs are thost.
Xiao et al (2021) has a list of species sold at the Huanan market. I would encourage you to read that list and suggest which animals you think are plausible, and I will tell you why they are not actually plausible.
Roko (the one infamous for the basilisk) has argued that what might have appeared as a shift in expert consensus is in fact "manufactured consensus." This is worth considering. Link: https://www.lesswrong.com/posts/bMxhrrkJdEormCcLt/brute-force-manufactured-consensus-is-hiding-the-crime-of | I also believe an algorithm somehow sorting, weighting evidence cleverly could solve the problem of "too much evidence." The CIA, in fact, at least has used one such algorithm, invented by Richard Heuer.
I have no horse in this particular race, but I do have a lot of expertise in some of the areas rootclaims "investigates" (especially the stuff related to Syria and chemical weapons) - where their analysis is so shoddy and laughable it's indistinguishable from Youtube conspiracies - and the biggest surprise to me here is that anybody really bothers with rootclaim in the first place? The more you learn...
How do you account for “one of the most powerful political actors in the world which has a history of controlling information flows runs the area in question and can be manipulating what evidence is available?”
I get that you have to work with the evidence you have. When the evidence you have is passing through a filter that has pretty strong incentive to hide its own wrongdoing (if that were to exist) how does that fit into your analysis?
Did they assess a probability that, ie, certain research records where the researchers tried that ~exact~ furin site were destroyed by the CCP? Wouldn’t that just be a “naked prior”?
I enjoyed this piece - I tried to follow this story in 2020 and have always leaned more zoonosis than lab leak (I believe I said 90-10 publicly in 2021, probably closer to 95-5 now).
Despite this, I've often been frustrated and annoyed by mainstream virologists who dismiss LL with poor reasoning, and almost always find myself defending (more reasonable) LL proponents.
I guess that this annoyance has led lots of people in rationalist-adjacent circles to lean further LL than they should have. I wonder if this is a common rationalist failure mode we need to update on.
I don't think this grapples with the problems the Mr Chen case poses for the zoonosis argument. Both sides in the debate agree that Mr Chen contracted symptoms on 16th Dec. This suggests he was exposed around 10th Dec. He thinks he was exposed on the train or in hospital on 8th Dec.
This raises two problems:
1. Mr Chen lived 30km away from the Huanan market on the south of the river, and never went anywhere near it in December, according to his interviews. His case was only recorded because he happened to be transferred to the top tier Wuhan Central Hospital, which is a sentinel hospital for respiratory outbreaks in Wuhan. It is in the north of the river from his local hospital because his relative happened to work there. This is evidence of a geographical bias in he case search. How many other cases were missed on the south of the river because they happened not to visit top tier hospitals, which were largely near the market.
Geographic bias in the case search is also confirmed by:
- Literal statements to the effect that 'we focused too much on the market' by the man in charge of the case search https://www.bbc.co.uk/sounds/play/m001ng7c
- A mountain of evidence in official Chinese documents of a market-bias in the case search. https://journals.asm.org/doi/full/10.1128/mbio.00313-23
- This short statistical argument -https://academic.oup.com/jrsssa/advance-article-abstract/doi/10.1093/jrsssa/qnae021/7632556?redirectedFrom=fulltext&login=false Cases with no links to the market were closer to the market than linked cases. This is evidence FOR ascertainment bias in the case search.
2. Given that there was a confirmed case 30km from the market around 10th of December, this suggests widespread community transmission across Wuhan in early December and late November. i.e. not that there was a zoonosis in early Dec. The first confirmed case was on 10th/11th Dec (even for that case, it is unclear whether the first case was market linked). There was no published retrospective case search. The first case was initially thought to be 1st Dec. Usually if you do contact tracing and a case search, the first official case goes backward in time. In this case, it went *forward* in time to 10th Dec.
My final comment. It is important to consider how different SARS-CoV-2 is to other zoonoses. I have a challenge for zoonosis proponents to find me a zoonosis with all of the following features
- Spillover occurred after 2000 when sequencing became much cheaper
- There were more than a hundred human cases
- There are zero infected animals.
This characterises SARS-CoV-2, but no other zoonosis meets these criteria. Why?
Thanks for your deep and well-reasoned dive into a truly important topic. For me, your most important take-home messages were a) Bayesian reasoning has serious practical limits; b) we need to take the potential dangers of viral lab leaks seriously; and c) we need to take the potential dangers of natural zoonotic spread very seriously. One point I haven't seen mentioned is that the Earth harbors roughly 10**27 virus particles.
My rule of thumb for something like this is 'whose evidence stands up to scrutiny'. I am not smart enough to understand any of this stuff, but I vaguely knew about the evidence beforehand from podcasts and social media. And then Peter just explained it all away effectively without significant pushback. This never happens in UFO debates. Or political conspiracies. Someone spends a billion hours researching sounds smart makes 30000 claims and then you pick 5 at random and 4 are nonsense and 1 is true or misleading or misinterpreted. I notice that my plausible theories of lab leak after hearing Peters argument about the implausability are *not* those advanced by rootclaim and so I assume they are obviously wrong I just don't know why but I'm confident Peter could convince me instantly. Rootclaim's inputs fell apart under scrutiny', which makes me doubt his statistical analysis. Garbage in garbage out. Note my real confidence about who is right is super low because I am dumb. But the debate definitely moved me away from lab leak 70-30
A lesson Scott draws is that Bayes is effectively impossible to apply in complex real-world situations, and that ultimately we are forced to rely on our intuitive reasoning. We’ll just throw a little math in at the end to make sure we aren’t making some clear and obvious blunder. I suggest this means the true epigraph for ACX should not be “P(A|B) = [P(A)*P(B|A)]/P(B), all the rest is commentary,” but rather “commentary, and all the rest is P(A|B) = [P(A)*P(B|A)]/P(B).”
> When psychoanalysts claim their therapies work, they don’t mean that someone who just read a two page “What Is Psychoanalysis?” pamphlet can do good therapy. They mean that someone who spent ten years training under someone who spent ten years training and so on in a lineage back to Freud can do good therapy.
And Robyn Dawes in House of Cards showed the evidence doesn't support the idea that training makes them any better.
“One read out of 200,000,000 is completely statistically insignificant,” said Pond. “It really had no SARS-CoV-2. There is no evidence based on genetic analysis there was SARS-CoV-2 in that sample. One read out of 200,000,000 — it could have been a low level of trace contamination.”
What’s more, as Bloom’s preprint reports, Q61 was the only swab above a certain threshold for raccoon dog genetic material that contained any SARS-CoV-2 RNA at all: “13 of the 14 samples with at least 20% of their chordate mitochondrial material from raccoon dogs contain no SARS-CoV-2 reads, and the other sample [swab Q61] contains just 1 of ~200,000,000 million reads mapping to SARS-CoV-2.” When Bloom plotted the quantity of animal genetic material found in the swabs with their SARS-CoV-2 RNA content, he determined that there was in fact a negative correlation between the abundance of SARS-CoV-2 and genetic material from raccoon dogs in the swabs.
Wow this was a great read.
But also.... What happened to discussion of "NIH-funded gain-of-function research" by Peter Daczak and EcoHealth alliance? Is this the "DEFUSE" grant I'm question? It makes not mention of University of North Carolina - that's new to me, though of course I didn't delve deep into the grant.
https://www.science.org/content/article/nih-restarts-bat-virus-grant-suspended-3-years-ago-trump.
I also wish it would be possible to go and collect all the Twitter posts on this topic from when the situation was unfolding, before and after the pandemic left China, including all sorts of claims: spread through plumbing, claims of corona virus circulating months (!) before the official first case, etc. -- do an analysis of how much is real, how much is fake. For example I remember following very closely everything I could read and don't remember racoon-dogs ever being mentioned, but pangolins were.
Can someone explain to me Peter’s argument "doubling rates at the wet market points at zoonosis rather than a superspreader event" ?
Intuitively, the two scenario (zoonosis / superpreader event) looks the same to me : an infected (lab worker/racoon dog) (goes at/is brought to) the market and infects market participants. I expect the same doubling rates in both cases.
Wow, this was incredible.
Is it really such a coincidence that the outbreak occurred naturally nearby the lab? It's reasonable (I guess, I don't know) that the lab was strategically placed there in the first place due to the area's wildlife, which host many animals relevant to their virology research. If so, the alleged coincidence significantly fades.
Thank you dashing young blogger!
I think the "lab leak" side of the debate focused too much on the probability that SARS-CoV-2 was engineered, and didn't properly account for the scenario that SARS-CoV-2 was a natural virus collected by virologists that accidentally leaked. In my opinion the latter scenario is more likely than the former.
The issues with extreme odds, focusing on the big picture, and eating one's losses are all related to the same phenomenon: The high complexity of the underlying processes.
Reality is not generated by a simple "zoonosis" factor competing against a "lab factor", as one would think if one naively took the simplest Bayesian models seriously. Rather there was a wet market and a lab and a bunch of people all doing a bunch of different things, etc.. Way too complex to explain. (Though because of funky math where we condition on the fact that there was a pandemic, we also can't ignore that there is a competition between these two factors; they just aren't the generators of reality, and therefore not of the evidence either.)
It is extremely common to have pieces of evidence with extremely low priors, because these pieces of evidence contain a lot of information about the specifics of these underlying processes, and probability drops exponentially with information. Notably, contra focusing on the big picture, accurately integrating this evidence requires building a narrow, specific picture of what happened, because otherwise you don't really know how to evaluate the interactions between the probabilities when conditioning on multiple things.
As you build these complex, low-probability models, that means you end up eating a bunch of losses, in rough proportion to the evidence you add to your model to account for the complexity. If your resulting model of what happened doesn't have very low prior probability, you're not thinking in enough detail.
Did either side get into the US intelligence assessments at all?
It seems to me that by far the strongest argument for lab leak is the US government conclusions to that effect. These agencies have an awful lot of raw data that no one else does, and the resources to put a lot of analytical effort into the problem. Obviously, they're fallible but they have access to the same scientific data as everyone else plus whatever kinds of useful classified raw data the US government has scooped up over the years and they employ experts on stuff. That's enough for a pretty decent presumption in their favor, especially when so much of the debate centers on data from Wuhan that could easily have been manipulated.
I'd find the zoonotic origins argument a lot stronger if someone from that side could explain what the FBI and DOE may have seen that led them to get it "wrong" based on some premise other than incompetence.
Am I the only one who Peter lost at the end when he starts asserting that any evidence that disagrees with him comes from liars? And his "evidence" that people are liars is stuff like "it was reported in the Daily Mail" and "he was a drug user" and "his cat got it too" (by which Reed obviously meant tested positive, which can absolutely happen to animals, inanimate objects, and all kinds of other random things). Followed by, "and everyone else was also a liar".
I have a friend who is convinced he caught COVID in Jan 2020 before anyone knew what it was or what to watch out for. He had symptoms that were an exact match. Officially, COVID did not exist here at that time. Is he also a liar? I assert that he is not - he has no reason to lie about this, and is an honest and upstanding gentleman. I therefore assign it a 1:10,000 chance that he is being deceptive, and thus COVID clearly spread far earlier than officially stated.
I also have a heuristic about people who dismiss evidence on the basis that it appeared in a tabloid, or wasn't in a peer reviewed papers: naive and gullible. Here's some Bayes for you: what's the probability that a journal publishes an article full of photoshopped images and garbled machine-generated text? My prior for this happening in an academic peer-reviewed journal is high, because there keep being incidents where whole editions of journals get retracted due to every article being gibberish. My prior for it happening in the Daily Mail or any tabloid is zero because it's never happened. Clearly, the Mail is far more reliable than peer reviewed journals and Peter should accept that its evidence overrules anything he's read in journals.
Rephrased: most of Peter's evidence implicitly assumes the testimony of China, virologists and epidemiologists is reliable (e.g. the ridiculously precise stated doubling period), when we know it's not. Many times he makes statements to which a good response might be, "and how do you know they aren't simply wrong/lying", but Saar was too generous to do that. Peter did not return that generosity!
Ultimately, although it was interesting and Peter's performance was impressive, I don't know if this kind of debate helps. I remain in camp Lab Leak.
As I've been saying since the whole debate started, whether or not it was a lab leak is really irrelevant. What we know for certain, with no actual controversy surrounding it, is far more interesting: China did not shut down international air travel out of Wuhan until significantly after they knew the virus was circulating.
While they were covering up the origins of the virus, and locking down Wuhan to try to keep it from spreading further internally, the CCP was knowingly, deliberately exporting this deadly plague to the rest of the world. And that is far more damning than any question of its origin. The origin of the virus is China. It deliberately inflicted it upon the rest of the world, which makes the pandemic a *de facto* bioweapon attack, whether or not the virus was developed in a lab with the intention of building a bioweapon.
If SARS-Cov-2 did not come from WIV then presumably WIV and the Chinese government would have had a very powerful motivation to open their records to outside investigators and make all of their people available for interviews. Instead they clammed up. Why?
I hadn't heard of Rootclaim before this, but that first screenshot which puts the odds of widespread fraud in the 2020 election at 8.7% tells me all I need to know -- that is orders of magnitude too high as anyone who understands how us elections are run would understand. Seems like there's something about this method, at least as practiced by Saar, that produces excessive probabilities of out of consensus or malicious explanations for things.
On the subject of this showing up baysian analysis, since viewers all tried to track it and got different results anyway:
Quant trading also works like this - it's basically using baysian inference for the stock market - and also ends up with a lot of big discrepancies and requiring a lot of Metis to get right. But it's still consistently significantly outperformed (in terms of real money earned in the real market) over traditional non-quanty trading. To the degree that baysian analysis does work in real life, this is what I'd expect it to look like.
"1/10,000 Wuhan citizens work at the wet market. So if a lab leak was going to show up somewhere random, the wet market was a 1/10,000 chance."
Why are we only counting the people who *worked* there? Wouldn't the relevant number be all the people who visited there (a much higher proportion of the population)?
First image: "The election was no different from previous elections" is not inconsistent with the claim that there was widespread fraud. As stated, none of the options are mutually exclusive.
(this is true regardless of the actual answer to the question)
I disagree with some of the arguments. I think COVID originally started spreading in rural villages. It didn't spread fast, because there wasn't that much unventilated space, and it wasn't well adapted to humans. And it wasn't detected, because until special tests were developed, COVID just looked like "atypical pneumonia". That is where it adapted to humans. Then it was carried into the wet market by a vendor. So it did spend some time learning how to spread in humans, but it wasn't by gain-of-function research.
Whichever debate participant is most well-versed is most likely to have thoroughly understood the topic and thus is most likely to have a correct intuition on the subject. Thus, we should just decide on these issues based on our subjective intuition about which debater seems to be most knowledgeable and rational, and not reason through the content of the debate except to fact-check the debaters and ensure their reasoning follows so they can't get away with intelligent-sounding bullshit. For example, instead of checking every single fact, just check a simple random sample of them. This approach may seem objectionable to some people, but it's probably what we're all subconsciously doing anyways, and it's probably approximately correct.
> this disease that came out of nowhere and ruined all of our lives for a few years
I wish we could have a debate on what the proper response to COVID should have been. In theory, if governments refused to do much other than developing vaccines, COVID would've had a significantly smaller impact on our lives. But also this would've also had significant costs to many people in the form of losing their elderly relatives and more people would've had long COVID.
I wish we could have a Rootclaim debate on what the Western countries should've done in late February/early March as a response to the virus. I think there's a large degree of disagreement about this question to this very day and it's definitely not been discussed enough in retrospective.
Awesome analysis. And really good that you picked up on the personality factor at the end. Kinda odd that Rootclaim do deep Bayesian stuff on the arguments, but do no personality screening for willingness to be objective / fear of losing a debate. The adversarial system is super-Western, I guess, with two dudes fighting their corners,. But in this instance it seems to me like it could be huge factor to swing things.
The comment at the beginning suggesting that it was too bad Putin doesn't have cancer felt a little odd to me, BTW.
You know, I looked at the "widespread fraud" one at the top, and it struck me that it didn't even consider that I would think was a much more likely hypothesis for it: That several big city political machines, acting independently and in parallel, jumped in an applied diverse methods of falsifying the outcomes, in just those places where such falsification was both (a) readily manageable and (b) critical to their respective states' electoral votes (giving motivation). It makes me suspect that you can get the results you want from Bayesian methods by carefully curating the initial hypotheses to compare. Not very different, perhaps, from C.S. Lewis's trichotomy about Jesus.
Hot take: Peter clearly failed to convince anyone.
The lab leak odds, in log10 (i.e. orders of magnitude are):
Peter -20.7
Saar 2.7
Eric -3.1
Will -2.5
Scott -1.2
Daniel -1.4
One of these numbers is clearly an outlier. Scott mentions it and calls it "trolling", I would argue that it is debating in bad faith. 2e-21 is a ratio which is just silly. For one thing, the gain of function at WiV pathway is not the only pathway towards a lab leak. The WIV could also have released a naturally occurring coronavirus at the wet market. At 2e-21 odds, we would probably have to consider the possibility that the WIV built a time machine and went back in time to infect the wet market.
Apart from the pangolin which got retconned out in favor of the raccoon dog, one thing I remember from the earlier discussions was that proponents of the lab leak pointed out that the bat with the virus likely did not come from Wuhan. Given that Saar has not made that argument, is that no longer the case? I imagine that by now we have a much better understanding of naturally occurring bat corona viruses in China.
Hi Scott,
thank you for mentioning my "preachy and annoying" blog. :P
Can't do much about my eyes rolling backward on this topic, but since you are all about evidence and probabilities; you might want to keep your eyes peeled for my next 15000-word sci comm article that comes out in two weeks and will bring people to the cutting edge of origin science, as well as put the "gain-of-function research" narrative to bed once and for all.
Promised :)
Cheers
> Fourth, for the first time it made me see the coronavirus as one of God’s biggest and funniest jokes.
The punchline is "hindsight is always 2020"
I was pretty sure about lab leak, now I'm 50/50. Normally I should be frustrated to end up 50/50 after reading all this but I'm filled with a feeling of satisfaction. Great job for all involved.
I am WAY out over my skis here…but it just seems like there is no objective way to actually TEST any of your priors in a scenario like this. So you’re just giving a finite number (and “false” sense of precision) to what actually remains a messy and murky quandary.
i "like" the Cosmic Joker hypothesis, meaning it's the most *interesting* one to me. i also like the idea of the Covid virus as a virus in a superposition, taking inspiration from Scott Aaronson's comment in a podcast that "There is a serious prospect that in our lifetimes people will be able to create superposition States of, let's say, a virus". (dunc tank podcast, July 11, 2020, with minor editing from his improv oral) in this krazy view, the two states of the virus are: (i) being from the Wuhan virology lab: and (ii) being from the wet market. the evidence never resolves decisively to one of them. to put it another way, a meta-version of the lab leak hypothesis is true: the Lab is some Schrodinger's Box, and the Covid virus escaped while still in its superpositional state. it moves though our world leaving a neat two-path evidential trace.
https://philarchive.org/rec/BALCDT-5
Very nice, thanks for watching, and reporting. I want to make an argument for agnosticism. It seems like we are all inclined to think we must have an opinion on almost everything. What is wrong with saying, "I don't know"? Being trained in physics, I just want more good data before I try and make any sort of decision. Eventually I hope we will hear news of some intermediate animal host, or news from some researcher that says, yeah we were doing GOF on those viruses. Until then I don't see how the odds (for me) can be anything other than 50/50. A sign of my almost complete ignorance*, and an unwillingness to give anymore credit to one sides 'experts' vs the others. I think the best path for humanity is to assume both things are true. And try to make both forms of possible virus mutation less likely to happen in the future.
Oh, and why are there no error bars in all of this analysis. I'm thinking the error bars should be huge.
*I read a lot of stuff about this in 2020, said then we need more good data, and stopped spending time thinking about it.
Thanks for the write up Scott! This is definitely very interesting.
I find myself walking away with a couple of thoughts. First, about the debate itself:
1) I started out pretty strongly leaning towards but far from certain about Lab Leak being correct. A lot of this I think stems from finding out that the anti-lab-leak side was a literal conspiracy that had a lot of institutional support to try to shut down the debate. Even those shouting down the theory secretly thought it might be true (at least early on)!
2) Likely related, I could feel learned helplessness setting in while reading the back and forth descriptions of the debate. With no priors on the believability of either participant, and seeing that both were making very strong claims that could reasonably answer their opponents, I want to mentally shut down and hold my existing perspective.
3) I end the debate slightly closer to zoonosis/no opinion, but still leaning towards lab leak.
Meta-discussion I feel much worse about Bayesian reasoning, especially the hard version with math. Peter clearly has a number that's beyond implausible, but the error bars on all of the numbers seem unworkably large. You could adjust all of the factors up and down by relatively small amounts and come up with any interpretation.
Even more, sometimes really really implausible things happen. As Scott notes here, something really implausible definitely did happen! In fact, across the world, implausible things happen quite often. People die of incredibly unlikely events all the time - an average of about 7 kids per year in the US have drowned in buckets since 1984. If you had never heard that number, what probability would you put on the chances of a child drowning in a bucket? There are 22.4 million kids 0-5 in the US right now, with seven per year drowning in a bucket, which suggests about 0.0000003125 chance of a child drowning in a bucket in a given year. No matter how you do the math, the chances are very truly low. But it happens seven times a year.
How would we model COVID's origins if something incredibly unlikely happened? It's not impossible that it originated in neither the Wuhan Institute nor the wet market. In the realm of possible, it may not even have started in Wuhan. Saying that something is *more* or *less* likely is all well and good, but reality happens in exactly one way, which can be the very rare event. I think 1/10,000 is incorrect for the likelihood here, but sometimes 1/10,000 things happen. Just because we say something is unlikely doesn't mean it's untrue. Saar implicitly believes that the wet market theory is ~6% likely to be true. 6% is far from ruling something out, even if he's arguing as if it's closer to 99.99% the other way.
Interesting debate, well covered.
I updated my lab leak confidence from 99% to 90%. It swayed but did not convince me due to the somewhat sketchy logic about wet market probabilities.
#1 I really enjoyed reading this.
#2 My beliefs shifted from pretty confident in lab leak to 50-50
#3 Reading through your summary, I often got this horrific feeling of college debate and dropping points. College debate has this concept of dropped arguments, where the worst thing you can do is not respond to a specific argument your opponent makes, because they automatically win that argument. That's what this felt like, I'm not actually convinced by Peter's overall story, I'm not even sure there's a coherent story there, but Peter kept coming up with specific situations where Saar was wrong and that convinced me.
I'm reading back through the lineages thing and, just reading through, Peters positive argument for why the older lineage started spreading later is pretty handwavy. Compare that to the next section where he goes through why the intermediates are a programming error, he's on point. But I come back to the core question of why the older lineage started spreading later than the new lineage and I don't have a great answer to the main point, and I think Saar's is better here, but Peter dominates on the subpoint.
I'm just scared of focusing too much on the subpoint and irrationally weighing that too heavily. I dunno? For people who watched it, did you get "debate bro"/Destiny vibes from Peter? That's what this kinda smells like, I notice intense debates should work but I could also watch 50 debate streams and at the end of it be dumber than I started and I felt this way a lot in college debate as well.
This was a wonderful read and personally seems like a great experiment. A bunch of credit goes to Saar and Rootclaim for courtesy and making this public and educating all of us, whatever position you hold.
> along with their $100,000 table stakes
I thought the judges fees came out of the stakes, rather than being in addition?
An insanely good debate in many ways, and congratulations to Scott on a most excellent summary. This reads like a communication from an alternate universe. Seriously, since when are there 15-hour highly structured debates at the object level with this level of intelligence brought to bear? That just does not happen in this world, except it just did.
Despite the outcome not being what Saar wanted, he has ably promoted Rootclaim as a useful tool. I want a lot more of whatever this is.
Despite Peter being one of the smartest people I've ever encountered online, arriving at an odds ratio of ~10^21 against lab leak is prima facie evidence that he has not correctly employed Bayesian reasoning. That sort of ratio is where you start getting "simultaneous miracles". I estimate that the chances of a *natural* zoonotic event, meaning one in which human action was not a causal factor, infecting its first patient *inside the virology lab*, as being *considerably* more likely than 10^21 to 1 against. I possibly could estimate the chances that I, personally, created COVID, had it released at the Huanan Seafood Market, and suppressed my memory of doing so, as 10^21 against. (By this I mean if I sat down and calculated how likely such an event was, 10^-21 is an answer I might obtain. In reality I would throw out any probability that low and group the event with "weird shit happens".) 10^21 is a sextillion. I can't count that high and neither can Peter.
Not getting answers like 10^21 touches on a real problem with practical Bayesian reasoning, which is that almost all independent events are only very nearly independent. Clinamina like the Self Memory Wiping Garage Biolab Operator Who Is Me exist in enough possible worlds to start having an effect if you multiply together enough large Bayes factors. This is far from disastrous to the project of being better Bayesian reasoners but there should be more effort going into developing and teaching the techniques that deal with it. Rootclaim looks like one of the places that might happen.
During the week of January 23rd, 2020, the day lockdowns were enforced, there was somewhere between 175-225 excess pneumonia deaths in the Wuhan DSP per 100K. This is eyeballable in figure 1 here.
https://www.bmj.com/content/372/bmj.n415
They report 243 per 100K for the entire first quarter of 2020. That's an *enormous* number.
Smooth this out over a week and say that during the week of 1/23/2020, ~30 more people than expected were dying from pneumonia in the Wuhan DSP per 100K.
The cool thing about the beginning of pandemics is that there's no immunity and the mixing assumptions of SIR models probably work well in the beginning of a respiratory outbreak. You can play with this one:
https://gabgoh.github.io/COVID/index.html
I couldn't find good information on the size of the Wuhan DSP, but allow it to vary between 2M and 10M people. Assume the R0 was between 3-4, the CFR was .5%-1%, and a few other now fairly well known things about the course of the disease. And look at the fatality numbers.
Using these ranges and standard compartment model logic, I don't see how you get such an enormous increase in excess pneumonia deaths consistent with an early December outbreak in the HSM. It doesn't make any sense at all. The best case scenario from these simulations is sometime early in November 2019.
Several of Peter's arguments treat the doubling rate as a real property at low populations. That doesn't seem right.
If patent zero only infects one person, the pandemic is delayed by one doubling period. If they infect 4, it arrives one period sooner. What are the relative odds between someone infecting zero, one, two, four, or fifty? When there are enough infected this kind of effect averages out and we can just say it doubles at some rate. At small numbers you get swings in infection progress based on how each person rolls.
Because of that attempts to extrapolate infected populations backward using the doubling rate seem doomed once you get into single digits. The error bars on that kind of estimate should be large. It's noisy.
So I don't think you can confidently say things like this
> The COVID pandemic doubles every 3.5 days. So if the first infection was much earlier - let’s say November 11 - we would expect 256x as much COVID as we actually saw
Or this
> Although Lineage A is evolutionarily older, Lineage B started spreading in humans first.
> We know this because Lineage B is more common. Throughout the early pandemic, until the D614G variant drove all other strains extinct, a consistent 2/3 of the cases were B, compared to 1/3 A. Both strains spread at the same rate, so the best explanation is that B started earlier than A. Since COVID doubles every 3-4 days, probably Lineage B started 3-4 days earlier than Lineage A, which explains why it’s always been twice as many cases.
This is silly but:
> This is a pointless too-clever-by-half “flourish” that there would be no reason for a human engineer to do.
Have you ever met an engineer? We do this kind of nonsense all the time! This particular argument cuts in exactly the opposite direction Peter intended.
I don't have any comment on the outcome, but this little detail drove me nuts.
Peter claims that the WIV sampling stopped in 2015. This is not true. See page 60 downwards of this FOIA. https://s3.documentcloud.org/documents/21170561/536974886-gain-of-function-communications-between-ecohealth-alliance-and-niaid.pdf
EcoHealth say this to NIH in 2018:
"As per last year, we will not be subco ntractin g any funds to the intuiti ons in these
countries. All efforts expended in these co untries will be from collaborating
partners and not funded by our award. PI, Co-Investigators or other team
members may con duct short field ttips to assess markets, identify wild life in
them, and anange for shipment of samp les of bats and other high-risk host species
in co untri es that nei ghbor China (Burma, Vietnam, Cambodia, Laos) and that
supply wildlife to the internationa l trad e to China (Tha iland , Malay sia,
Indon esja)."
i.e. they say in 2017, they did sampling in these countries, and all samples would be sent to the WIV. Weirdly, all of the closest known relatives to SARS-CoV-2 have been found in places the WIV was actively sampling for more than a decade.
This is not even considering the sampling that the WIV would have done without EcoHealth funding.
The DEFUSE proposal proposes extensive bat harvesting in Yunnan in 2018. This is also evidence against them having given up on bat harvesting in 2015.
My intuition is that Rootclaim is correct that you should be able to use Bayesian spreadsheets--especially after some adversarial collaboration--and get a format that consistently outperforms intuition and even professionals informally weighing the evidence. It's sort of like how as an analyst I use fermi-calculations to get much more reasonable estimates of unknowns than just trying to estimate the unknown directly. I think Rootclaim should focus more on testing that.
I'm with you (after reading two long summaries of their debate) that zoological is more probable but also that Rootclaim is a worthy project. I wonder if the change in format that Saar should focus on though is more emphasis on the spreadsheet and adversarial collaboration to get both participants using the same spreadsheet. I'd love to see a version of the spreadsheet that two smart opposing viewpoints people agree is a reasonable setup (without any multi-stage fallacies, etc.) so we can see which line items are driving most of their disagreement free of any logical reasoning errors (or at least free of errors that adversarial smart people could identify). And then judges can focus more on the individual ratios (and any remaining spreadsheet steps that debaters couldn't adversarially reconcile)
That was really interesting. My bayes argument would be something like "If covid WAS leaked from the Wuhan research lab, how plausible is that picture of events?" If so I'd expect an average amount of "well, we don't know" for scientific research. But each thing that needs to be explained away rapidly adds to "that doesn't sound right". Today I learned that they did do modifications like this. But I also learned that they were usually publishing their research like any academic institution, so you need coincidences or cover-up to suppress anything they'd done on COVID specifically.
This includes a lot of things that I don't know. But I still like my reasoning, because I know there's lots of biologists out there who have a better picture than me. And I haven't heard of any reason for biologists to avoid thinking about a lab leak, and don't think it's likely there could be a reason that wouldn't be well-known. (Whereas there ARE reasons for physicists to dislike many worlds, or doctors to dislike "chronic fatigue exists and easy weight loss doesn't") But what I usually hear from random biologists is "well, yeah, it could be, but I don't have any particular reason to think it is". And so I think that's probably right.
I'm interested in the ways of putting more specific numbers into reasoning but I think you can rapidly get into territory where it's hard to tell if the reasoning is sound. E.g. trying to describe how much of a coincidence each theory is raises questions like, if it was zoontic, how many other places would have sounded as suspicious as the wuhan research lab? Restricted secret areas? Other kinds of labs? How unique is the nearby lab, in advance? If it was a leak, how many places would have seemed as suspicious as a wet market? Nowhere else seems very suspicious, but "coming out of nowhere" would have seemed completely plausible for zoonotism. Or how much are the lab and the pandemic both there because there's convenient reservoirs of coronoviruses nearby, that could be, I've no idea. I think trying to make these precise leads to "things we didn't know we needed to account for" swamping the analysis.
I ought to read how the probability calculation is actually working. I would be really interested. Although I'm not sure it would help: e.g. I'm more persuaded by my argument of listening to biologists, so any attempt to quantify likelihoods for genetic arguments will be swamped by the chance of something I don't know mattering.
Perhaps this description of the debate wrongly magnifies this, but it struck me as really discrediting to the lab leak side that they kept relying on debunked claims. The LL side would bring up things lime the 90 missing cases, the daily mail guy, or the idea that the market was a superspreader event, and Peter would describe that evidence in greater depth and be able to pinpoint why that evidence was faulty, which was not rebutted by the LL side. One of the most annoying habits conspiracy-adjacent thinkers online have is to gish-gallop between weak evidence they're only lightly acquainted with, and then not change anything about their thinking when a piece of evidence is disproven. I generally find the meta-bayes "what about the odds your argument is secretly bad and mine is secretly good" mostly annoying, but how can you not apply a pretty heavy discount to the credibility to a side that is frequently trotting out rather flimsy evidence?
The $100K prize introduces a huge signaling hazard, mainly because Saar is rich and Peter isn't. I'm sure Saar's lost that much playing poker. For Peter, this is a sizable portion of his net worth — so he plays to win (and, as other commenters have pointed out, may be playing a bit dirty).
That said, the biggest issue with this debate format is that it's about an *incredibly important topic* where several major players have *a lot* of skin in the game. As much as this is a battle of evidence, it's also a battle for narrative (see Peter's rationale for wanting to get one up on the conspiracy theorists). And the "Official" narrative, that all good, sensible, educated people should agree on (you don't want to be a conspiracy theorist, do you?) is zoonosis. There are probably better, less charged questions where the Rootclaim Bayesian approach would work.
Not that I'm entirely convinced by Lab Leak either - maybe 60% odds? But given that I read ACX while goofing off at work, I don't have time to watch 15 hours of debates. Gotta go with my gut at some point.
Two big issues with trying to formalize Bayesian reasoning is to properly identify what is already part of your priors and what isn't. Second, using it at scale in a systematic way so that you actually benefit from the correct updating mechanism. After all, the more you just rely on the prior the less it makes a difference.
The problem is that we only pay attention to issues once something makes us think there is an interesting question there and often these issues are relatively unique so we have no obvious choice of base rate to use as our prior. And there is the very real danger of pulling in information that is really only a result of the update.
I think distinguishing the prior is one place where AI can sorta help. Maybe not yet, but a great feature with AI is that you can remove data from it's knowledge and see how it would judge things. I'm particularly hopeful for this in politics (this AI predicts your judgement in these political issues 90% of the time and when it doesn't know that Trump supports this policy it predicts you will too).
I'm more skeptical about the ability to do this systemically at scale in a way that uses enough interconnections to improve judgement.
Also a statistical analysis that favors bass sounds awesome!
I think people get caught up about the furin cleavage site mutation because they're looking at the wrong base rates. To think about the furin cleavage motif, you need to ask (I'm phrasing these questions a bit imprecisely):
- How often are furin cleavage motifs studied in places other than Wuhan? How often are they studied in Wuhan?
- How often are furin cleavage site insertions the cause of viruses learning how to become pandemic? How often are other mutations the cause of viruses learning how to become pandemic?
I don't see people asking those questions. The fact that people don't ask those questions makes me suspicious of everyone involved, honestly. These appear to me to be the questions that arise out of applying Bayesian reasoning straightforwardly. (I'm also suspicious, as a sidenote, because the question overall is kind of irrelevant: everyone agrees lab leaks can cause pandemics, and everyone agrees zoonosis can cause pandemics, and everyone agrees zoonosis is more common than lab leaks, so we should just try to prevent both, focusing more on zoonosis than lab leaks.)
Also, WTF do people care so much if this particular disease originated from a lab leak.
We have a pretty good grip on the problem of biolab safety (eg the leaks in the UK) and the existence of gain of function research. So whether or not COVID itself happened to be a lab leak isn't reason to update too much on the future probability. Any forward looking policy question shouldn't really depend much on whether COVID was or wasn't a lab leak.
I get the debate is interesting but it's ironic that a claim to be more rational is being tested via a subject that it's irrational to be so concerned about.
My favorite post in a long time. Thanks for the recap Scott.
"Either a zoonotic virus crossed over to humans fifteen miles from the biggest coronavirus laboratory in the Eastern Hemisphere. Or a lab leak virus first rose to public attention right near a raccoon-dog stall in a wet market."
One thing that strikes me is that there is an asymmetry here. The first part of the above does indeed sound like a grand coincidence. It's not like the mutating virus would know that it happened to mutate near a lab. However, the second part is only a grand coincidence if we assume that if there was a lab leak, that it was a) fully accidental, and that b) no one acting in bad faith had the ability to distort data from early testing, etc..
I'm not sure I'm of the opinion that lab leak is thus more likely. But I can't help but think: it sure would be convenient if you were a lab researcher/administrator/related government official and you didn't want to be blamed for something you negligently made possible - then it sure is convenient that there is that nice big wet market nearby that perhaps you can shift the blame toward.
My current thoughts: though I didn't sit through the 15 hours - I am now of the opinion that the zoonotic hypothesis is stronger than I had thought before. I am updating to be less confident in the lab leak. But I'm probably now to about even odds, fwiw.
Michael Weissman's analysis is by far the most principled and persuasive thing I've read on the topic, it deserves much more discussion than just the link.
The claim that you shouldn't have high Bayes factors on both sides of a debate is just false.
Every event with sufficient detail involves a ton of unlikely stuff happening. Every time you flip 30 coins you get an exceedingly unlikely list of heads and tails.
Sure, if you understand the data to be the **complete** specification of the even then yes you don't get high Bayes factors on both sides because you only get one. But that's not what's going on here. They are both cherry picking the aspects of the situation that most favor their hypothesis so of course you get high Bayes factors on both sides.
I don't have anything to add on the COVID origins topic, but I do want to register some (possible?) disagreement with this bit:
"For example, suppose I win the lottery, I’m told I win the lottery, the lottery company gives me a big check, I cash the check, and I become rich. Given that there were 1-in-100-million odds against me winning the lottery, the lottery company giving me the check and so on must be at least 100-million-to-1-level evidence - otherwise I should refuse to believe I really won the lottery, even as I enjoy my newfound wealth!"
The quip-level counterpoint: You *shouldn't* believe that you won the lottery, because which is more likely -- that I won the lottery and cashed the check and got all this claimed evidence, or that I'm *dreaming* that I won the lottery and cashed the check and all the rest? After all, I've had several dreams where I won the lottery before, and I can get just about any evidence you like in a dream (all made up by my brain, of course). Sure, a 10^8:1 update is possible in this case (er, maybe, I'm not sure because it sort of hinges on things I think I know about dreams but which might be false -- after all, I might be dreaming I know them), but the strongest counter hypothesis is this sort of out of band thing (dreaming, or maybe something like solipsism or a "living in a simulation" or "being in The Matrix" or similar roughly-equivalent thing) rather than something like an elaborate prank or a mistake.
Yes, on at least one occasion I have realized that I was dreaming by applying this logic.
This might seem academic or silly -- how do I **really know** that I'm not dreaming my whole life, being continually deceived by Descartes's demon, etc. -- but I'm pretty sure that similar caveats ought to apply to trying to reason about some things that people care about, like "Is there a God?". If you think you are more than 99% sure you know the answer based on Bayesian reasoning, you are being overconfident that you (a) understand the various conditional probabilities involved and/or (b) have properly considered all the relevant hypotheses.
What’s the Bayesian probability that Scott wrote this entire article to make the pun in the first section heading?
> So probably we should just accept that the first reported case - a wet market vendor, December 11
So, after searching a bit, the closest article I found is this : https://www.nydailynews.com/2020/03/27/shrimp-vendor-identified-as-possible-coronavirus-patient-zero-leaked-document-says/
A shrimp vendor, not raccoon dogs vendor. This is to me an important piece of information and leaving it out feels very misleading.
I don't want to divert attention from the debate, but since it seems appropriate, I'd like to share a post I wrote nearly three years ago proposing an hypothesis that is not exactly lab-leak, though not incompatible with it, and not zoonosis (at least as I've seen it stated):
https://medium.com/analytics-vidhya/a-novel-hypothesis-previous-ancestor-virus-ba2ae7b10062
Short version:
The propagation of Covid-19 seems to point to certain areas of the world being less affected by the disease for no particular reason (no younger population structure, no less obesity, etc...). The main area being Southern China and Northern SEA. This is precisely the area where you would expect zoonosis for a bat coronavirus to happen. Other geographic areas correspond to places where infected people might have first, and in greater numbers, arrived if they had been infected by a previous virus in the aforementioned main area. This and other evidence (see full post) point to the possibility that a previous ancestor (to SARSCov2) virus emerged and propagated in that area, before mutating into SARSCoV2.
The competing 1/10,000 claims about the wet market are interesting, but feel too speculative without enough hard evidence for me to know what to do with. Far more salient and intriguing to me are:
Saar's claim that the 12-nucleotide furin site comes out of nowhere...
> COVID - which mostly just resembles BANAL-52 with a few scattered single-point mutations - has twelve completely new nucleotides in a row - a fully formed furin cleavage site that came out of nowhere.
vs. Peter's claim that the the furin site doesn't use the expected sequences for such a site and actually uses sequences that wouldn't be expected to work. Plus Peter's claim that WIV didn't have the technology or expertise to insert such a site anyway
> COVID’s furin cleavage site is admittedly unusual. But it’s unusual in a way that looks natural rather than man-made. Labs don’t usually add furin cleavage sites through nucleotide insertions (they usually mutate what’s already there).... COVID’s furin cleavage site is a mess. When humans are inserting furin cleavage sites into viruses for gain-of-function, the standard practice is RRKR, a very nice and simple furin cleavage site which works well. COVID uses PRRAR, a bizarre furin cleavage site which no human has ever used before, and which virologists expected to work poorly. They later found that an adjacent part of COVID’s genome twisted the protein in an unusual way that allowed PRRAR to be a viable furin cleavage site, but this discovery took a lot of computer power, and was only made after COVID became important. The Wuhan virologists supposedly doing gain-of-function research on COVID shouldn’t have known this would work. Why didn’t they just use the standard RRKR site, which would have worked better? Everyone thinks it works better! Even the virus eventually decided it worked better - sometime during the course of the pandemic, it mutated away from its weird PRRAR furin cleavage site towards a more normal form.
Peter's claims seem stronger to me, and I didn't see any response to these specific claims in the summary (although I may have missed), but I also can't explain Saar's claim, nor do I have the expertise to assign probabilities to any of this. But these claims seem to me to be the most important and what I would love to see further discussion about.
typo: they a found
I just wanted to respond to this point in the conclusion:
"Cool Machiavellian plot you have there, but maybe the fact that you’re losing 16%-66% should make you question whether you’re really as smart as you think you are...At some point you have to start debating!"
The reason that people in the field might be reluctant to accept this point is that they don't, and shouldn't, give a shit what 66% of the American public think. And debating just gives the American public the impression that it has some kind of a right to have a say in what the truth is. If you're a virus researcher now, working on how to predict and prevent the next zoonotic infectious epidemic, you really really don't want the public deciding to shut you down or order you to focus on lab leak prevention.
This is obviously very problematic - I'm kinda making the argument for important things to be decided in smoke-filled back rooms. But it's also unavoidable, I think. Science must by its nature be difficult and cutting-edge and unknowable to most people; and scientific advances are by definition decided by something other than human opinion - ideally, by what the universe is really like. So there is just an inherent limit on the level to which science can be democratised. Which means that noting which way public opinion blows is unlikely to be very relevant to the people who want to advance science.
That said, I do think that science is getting more democratic, and that that's a good thing. It's just a messy process.
I'm against using a text format for future long-form debates like this.
I watched this debate an a second monitor while doing other non-cognitively-demanding things like working on design projects and playing video games- and I enjoyed it a lot more than the podcasts and audiobooks I usually use for that purpose. In contrast to those, nothing about the debate felt like dull filler, and yet, unlike most really demanding books, it was possible to miss bits here and there without losing the overall thread of the argument. It's almost a perfect format for content to listen to while slightly distracted, and I think that for that reason, debates like this can provide a ton of value to people beyond just shifting their opinions on the topic.
Granted, a text format would probably result in slightly better arguments- though I don't buy that Peter ambushing Saar with new claims was a major factor in his loss; the lab leak side had plenty of time to research claims between debates, and plenty of time to respond later. A book-length written debate, however, is something I expect almost nobody would be willing to read, since it doesn't provide that extra value as good content to consume while doing other things. It would also require a lot more effort on the part of the debators, so I'd expect there to be both fewer long-form debates and much less engagement- and I don't think a slight bump in quality would at all make up for that loss of value.
Fantastic post - I've shifted my opinion quite a bit in response.
One thing that sticks for me is this: it didn't take long to find the civets responsible for the earlier SARS outbreaks. Why has this proved so challenging here?
The troll in me wants the answer to be that it was zoonosis that occurred in a lab, and then it leaked out of the lab.
Thank you for this excellent post. It's one of the best things I've read on the internet ever.
I remain in the Lab Leak camp, where I started. I'm of course not 100% certain, and I don't think it's possible evidence would ever emerge that would make me 100% certain. I really don't like people who claim Lab Leak is a "conspiracy theory" and am inclined to discount everything they say.
A question:
Should we accept claims by the Chinese Government and technical/academic/policy experts in this field at face value? Aren't they all hopelessly conflicted? Even if what they say is logically sound, aren't there myriad ways in which they could be fudging the analysis a bit, even unknowingly? What value have researchers doing gain-of-function work on viruses provided to the world? What is the likelihood someone in the field is going to publish findings saying the risk reward in their field just isn't worth it and all their skills are worthless because the work should be shut down?
Another question:
What do those maps of the outbreak in the wet market purport to show? The location where someone was exposed to COVID (how can that be known)? Some point that a person who got infected passed through (what about all the other locations they visited)? Whatever they claim to be showing, how accurate could their measure be? Based on interviews? How hard would it be to fudge the data a bit - for example, an infected person tells a researcher all the places they've been in the last week, and of the dozens of locations the researcher chooses the wet market to record.
First of all- awesome post! I loved all of this. Saar for setting up this challenge, Peter for taking him up on it, the judges for being fair, and you for summarizing the results- all great stuff.
As for the weird coincidence of the Wuhan Institute of Virology being located so close to the epicenter of the breakout- is it? Everyone knew that wet markets were a potential risk center for spreading diseases. See for example: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7141584/ about the 2003 Avian Flue outbreak. That's the same year that China approved construction of a biosafety level 4 lab at WIV.
They built the lab there because it was a good place to gather research data, and maybe also because the local government was strongly interested in that sort of research to protect themselves. It's the same reason that the "Center for the Study of Active Volcanoes" is in Hawaii, the "Earthquake Engineering Research Institute" is in California, and the "Hurricane Resilience Research Institute" is in Houston. Of *course* they are there where the threat is greatest. And gain-of-function research was just the most effective means of researching that kind of virus, if you want to research it before it actually starts spreading.
You might ask, if they knew the threat of wet markets, why didn't they just shut them down? My understanding is that China doesn't have that kind of state power. There are too many poor, rural people who really want to eat meat, and still can't afford ranch-raised animals. If you try and shut down the wet markets they'll just start selling them in secret someplace even worse.
seems like we should put some thought into hypotheses that prevent the coincidence
could Chinese authorities manipulate the evidence to have a wet market excuse?
maybe the raccoon-dogs intentionally sent their trojan dog to Wuhan to frame the virologists
Nitpick on that poll asking Americans if they think lab leak is true. (Before this) I'd have had to answer this as "probably true" but my probably was that I was at like 60% lab leak and 40% zoonosis. 40% chance of zoonosis is still pretty dang high and doesn't really seem that indicative of acceptance of the bad epistemic edifice that accreted around lab leak. I mean 30%ish saying it's "definitely true" is a problem but "probably true" seems to be doing a lot of work.
Also, good post. Quite markedly shifted my opinion.
Another looooooong series of posts that markedly shifted my opinion on a controversy with a lot of noise recently was this series:https://radleybalko.substack.com/p/the-retconning-of-george-floyd
*Edit* Also, both these posts and that parliamentary procedure kid that's been mentioned so much recently has made me think that the best model for thinking we have isn't heuristics or Bayesian reasoning or the wisdom of crowds. It's probably just smart guy who does like a ludicrously unreasonable amount of homework.
I think this quote stood out to me as off model:
> Zero COVID era Chinese outbreaks were concentrated in wet markets because they received infected animal products. We know why there was an outbreak in the Xinfadi Market in Beijing: it was because the seafood stall got frozen fish from some non-Zero-COVID country, the fish had COVID particles on it, and the vendor got infected and spread it to everyone else. Something like this is true for the other Chinese wet market based outbreaks we know about it. So this makes the opposite point you think it does: wet markets start outbreaks because there are infected goods being sold there. Then the virus spreads through the wet market at a completely normal rate.
I definitely know that China ascribed COVID outbreaks to things like "the person in the plane on the previous flight left covid residue". China was desperate to explain every outbreak so they could continue attempting to achieve a horrible and impossible goal. I don't actually believe that is how the virus was spreading? In particular, surface contact spread and the idea that you should wet wipe your boxes quickly fell out of style. I think the ubiquity of spreads getting first observed at wet markets during covid zero is strong evidence that even if covid is zoonotic but first entered the population by like, bats during a vacation, you would still have first observed it in the wet market.
I filled this out with others' hidden, and ended up weighing that even less than Saar did, which probably means I'm overly dismissive.
https://docs.google.com/spreadsheets/d/1IOHp6PINEbqizpbO7liXLEQTsIuNRpqx/edit?usp=sharing&ouid=113816084183231439418&rtpof=true&sd=true
I hope this isn't a derail: In one of the previous ACX posts' comments, someone posted a link to a page explaining the case for gain-of-function research. I'm embarrassed that I didn't either follow the link or bookmark it. Does anyone have that link? Many Thanks!
Way back in 2020 there was an article https://www.independentsciencenews.org/commentaries/a-proposed-origin-for-sars-cov-2-and-the-covid-19-pandemic/ , which I read after George Church tweeted it (!) (without comment or explanation). Their proposal (they call it "Mojiang Miner Passage" theory) in brief was that it WAS a lab leak but NOT gain-of-function. Rather, in April 2012, six workers in a "Mojiang mine fell ill from a mystery illness while removing bat faeces. Three of the six subsequently died." Their symptoms were a perfect match to COVID, and two were very sick for more than four months.
The proposal is that the virus spent those four months adapting to life in human lungs, including (presumably) evolving the furin cleavage site. And then (this is also well-documented) samples from these miners were sent to WIV. The proposed theory is that those samples were put in a freezer at WIV for a few years while WIV was constructing some new lab facilities, and then in 2019 researchers pulled out those samples for study and infected themselves.
I like that theory! I’ve like it ever since 2020! It seems to explain many of the contradictions brought up by both sides of this debate—it’s compaible with Saar’s claim that the furin cleavage site is very different from what’s in nature and seems specifically adapted to humans, but it’s also compatible with Peter’s claim that the furin cleavage site looks weird and evolved. It’s compatible with Saar’s claim that WIV is suspiciously close to the source of the outbreak, but it’s also compatible with Peter’s claim that WIV might not have been set up to do serious GoF studies. It’s compatible with the data on differences between COVID and other previously-known viruses. Etc. (As I understand it.)
Old as this theory is, the authors are still pushing it and they claim that it’s consistent with all the evidence that’s come out since then (see https://jonathanlatham.net/ ). But I’m sure not remotely an expert, and would be interested if anyone has opinions about this. I’m still confused why it’s never been much discussed.
This was incredibly informative. Thanks to the participants for doing it, and to Scott for summarizing and publicizing it. I hadn't thought about COVID origins in a while, but I would have told you it was in the tossup zone, and digging into this a bit puts me closer to Scott's 90-10.
I wish this was a highly replicable method for illuminating tricky topics, but I kind of suspect that the conditions here (two reasonable and sincere people who believed an impartial jury of smart, curious neutral observers would be won over to their side, and who were able to agree on who constituted a smart curious neutral observer) are tough to arrive at. I hope I'm wrong!
"some rando who nobody had ever heard of"
Not to be too facetious, but perhaps Rootclaim should have calculated the probability of the existence of a debate savant before building an attractor for one.
The big problem is that we still have very few details about all this. How is that possible after all these years?
The Chinese authorities haven't been very transparent. Other governments haven't demanded transparency from China. Why not? They just accept that China doesn't want to give any details.
The pandemic was a global catastrophe. It killed a ton of people and caused enormous economic losses.
The people deserve to know how it all happened. Secrecy is unacceptable.
The main reason why I'm leaning towards the lab leak is the severe censorship about the topic. That came from the individuals who strongly supported the zoonosis hypothesis. Usually people who desperately want to hide something have something to hide.
“Hey, this bat we just collected on one of periodic field trips to yunnan had an interesting viral strain with some really interesting structure around how it’s furin site works and implications for protein folding”
“We should probably research it, is there any grant money we can use?”
“I think the Americans just announced some new initiatives?”
“Great, let’s apply. Write a proposal that says we will investigate Furin site placement and it’s impact on virology.”
….
“Did we get the funding?”
“No”
“But the grant reviewers didn’t know about the bat, right? Just that we proposed doing the furin site stuff?”
“Yea, we’ve changed the names of the ones we found and generally obfuscated that we have this specific one in house”
“Should we just do the research anyway?”
“… … … yea, probably, could be interesting”
“We haven’t really done this before, and maybe we don’t have the right safety procedures and protocols, specifically to make sure it doesn’t escape?”
“No. But this particular strain can’t even last that long in the lab on its own. So what’s the worst that could happen”
I have no strong opinion about the Lab Leak hypothesis. I have a very strong opinion that Bayesian reasoning does not, in practice, help with deciding such matters.
Early in the post, Scott offers the following. After reading the entire long post, I believe that it is, in fact, the correct overall summary:
> But the joke goes that you do Bayesian reasoning by doing normal reasoning while muttering “Bayes, Bayes, Bayes” under your breath. Nobody - not the statisticians, not Nate Silver, certainly not me - tries to do full Bayesian reasoning on fuzzy real-world problems. They’d be too hard to model. You’d make some philosophical mistake converting the situation into numbers, then end up much worse off than if you’d tried normal human intuition.
Well I’m still convinced it was a lab leak but at least now I know I’m probably wrong.
Disclaimer: I have not watched the full videos. Apologies if this point is addressed at length.
To what extent are both parties relying on CCP-curated information for their data on the early cases? Was there a discussion of how (un) trustworthy that information might be?
If you assign 1/10,000 probability of that particular wet market being the location of the first major outbreak conditional on the lab leak hypothesis, you should also calculate the probability of that particular location under the zoonotic origin hypothesis. The population of Wuhan is <1% of the population of China (maybe the whole world is a better reference class, though rural areas and developed countries are going to have lower risk), for example, and that’s before you account for the fact that zoonotic origin could also cause major outbreaks at other locations within each city. All in all I’m not sure “covid began to break out at this particular wet market near WIV” is terribly strong evidence one way or the other.
Another piece of evidence I don’t see discussed here is the lack of openness from China, and relatedly the lack of evidence of a zoonotic source. If China had information against the lab leak hypothesis they would likely release it, so the relative lack of information we have is more likely given a lab leak imo.
For all the talk of Furin cleavage sites, there's one thing that confuses me. Could we have had a (far less deadly, probably not even noticed) coronavirus pandemic if there was no Furin cleavage site? My understanding of gain of function here is that there are multiple proteases that can do the cleaving even without Furin, but Furin does the job faster, and therefore sometimes outruns the immune system in a way COVID wouldn't do without that extra cleavage site. But that raises the question of why we got an outbreak of a virus with that Furin cleavage site, instead of a hypothetical precursor that lacks it. The existence of a Furin cleavage site is considered unusual, so for Zoonosis would you expect a precursor virus without it to be far more likely to cross over into humans first, and therefore prevent COVID from being a major issue in the first place?
Sometimes you can tell one of Scott's posts is going to be a banger just from the quality of the pun in the opening section header alone
Unfortunately it took me 16 hours to read this article 🥺
The notorious fact is that there were only 3 labs in the world at that time doing gain of function technology in the world...2 of those were im US and 3rd one in Wuhan..GOF was banned by Obama administration so the only destination to outsource technology was wuhan lab...🤗🤭 even famous defuse grant for the almost same virus creation by GOF was proposed and refused for wuhan lab by darpa im 2018...immpossible coincidencenes happen😅🤣
"Still, it’s awkward to use “conspiracy theory” as an insult when the conspiracies were real. Maybe a better question is whether lab leak is “pseudoscience”.
The argument against: lots of smart people and experts believed it was a lab leak. There were all those virologists giving 50-50 odds in their internal conversations. Even Peter says he started out leaning lab leak, back in 2021 when everyone was talking about it.
The argument in favor: since 2021, experts (and Peter) have shifted pretty far in favor of zoonosis. They’ve been convinced by new work - the identification of early cases, the wet market surveys, the genetic analysis. "
I have admittedly not looked into the details, and I cannot claim to truly know how to handle filtered evidence, but:
If the experts (mostly) all agreed to have a conspiracy pushing for A, and then all the evidence that turned up after that pointed to A, this really doesn't make me feel like I have to raise my probability of A all that much. For all *I* know for every published result supporting A there are 2 unpublished results supporting B, or claims that have not been investigated because experts (correctly) believe they would turn out to support B.
One issue I have with the opening presentation in the first video (I have watched 38 minutes so far) is that he lists, for example, about a dozen different theories about how the virus could have been made, and his slide says, "it's not possible for all these theories to be correct." So what? If there's one true explanation for something, the fact that 11 other people came up with a false explanation for it doesn't diminish the probability of the first explanation being true. Otherwise, I could make the probability of any true idea go down just by creating a bunch of false theories.
Skipped the whole lot and am working purely from Scott's post-transcript summary.
>So the real p(wet market|lab leak) isn’t the 1/10,000 chance a pandemic arising in a random place hits the wet market, but the (higher?) probability that there’s something wrong with Peter’s argument.
This looks like an excuse to assign whatever numbers you like to any point. If you're readjusting part of your stats, surely you should be readjusting all the other parts with the same readjustment metric, and thus reach a conclusion indistinguishable from the original formula.
It does sound like Saar lost mostly because he got caught flat-footed, but considering he set up the system it still looks bad for his methodology. Surely he used it to calculate a proper debate setup beforehand, and it underperformed.
I learned more about this debate and the arguments in favor of each side in the hour and a half reading this post than I did in any previous discussion of these arguments. Thank you for the rich summary, and while I agree with your arguments about why this kind of thing probably can't happen very often, I really wish we'd see more of these kinds of rich debates
A systemized version of this plus community notes is my basic model for how all news should work. But with reputation scoring in there to smooth out some of these transaction costs for actors who have proven themselves credible.
“Dumb” question that I did not see answered anywhere, and not mentioned much at all even in these comments to date: does “lab leak” also incorporate the possibility that it was created in a lab and *deliberately* circulated in the Chinese public?
And if done deliberately is an option within “lab leak”, doesn’t that throw off the odds by several OOMs, given that for someone deliberately wanting the virus out there, but wanting to cover their tracks a bit, “releasing” it in a wet market that had all those animals would be a fairly high probability thing to do (in addition to being a very good way to ensure spread)?
Is deliberately put out into the world its own separate possibility, or is it incorporated into “lab leak”? And if the latter, what is faulty about my reasoning that it throws off the supposed unlikelihood of the outbreaks being centered in the wet market by many OOMs?
Surprised to see no mention of Nick Patterson's latest: https://npatterson.substack.com/p/yet-more-on-covid-origins
I loved this whole thing, thank you for writing it and I'm glad you enjoyed it!
I have no dog in this hunt and don't feel like any important policy-making we have to do hinges on this outcome. We know we need to keep labs safe and we know we need to deal with zoonotic risks. And I still found it riveting to read, as well as all the comments here. I think this lands in the genre of competency porn and I'm here for it.
A meta-observation: I started today 50-50 on this question. Right after reading it, I was briefly at 90-10 zoonotic. An hour or two later, I'd say it slipped to 75-25, and now I'm landing maybe at 60-40. That's an interesting experience just to watch over the course of a day. I don't know, but I think if I read it all over again, it might goose me back up to 80-20, temporarily.
When I ask myself why looking at the debate as presented swings me a lot but then that regresses over hours, I would summarize that as "mistrust". Mistrust of the institutions that produce information, mistrust of the arguments presented, mistrust of the biasing potential of the debate format. That's my emotional/psychological bias and so the further I get from direct contact with the elements of the debate (which I found persuasive), then that default bias starts to take over more.
While my intuition listens along to the content of the debate, it also has some deeply-held opinions about the trustworthiness (or lack thereof) of experts. I have my hobbyhorse example of this around guidance to parents for what position to put their infants down to sleep -- which has changed three times since my adulthood. It's not that anyone was misleading anyone, but that things that look absolutely true to science in one era turn out to be absolutely wrong in the next era and that's how science proceeds. We don't know what we don't know, and that makes all my reactions to these kinds of competing stories pretty uncertain. And it also makes me marvel at other people's certainty.
I don't know, but I think this might have some relevance for questions people ask about how to persuade people about the importance of large scale risks based on a mish-mash of reasoning and data. Where and how mistrust shows up is a huge issue.
> Fourth, for the first time it made me see the coronavirus as one of God’s biggest and funniest jokes. Think about it. Either a zoonotic virus crossed over to humans fifteen miles from the biggest coronavirus laboratory in the Eastern Hemisphere. Or a lab leak virus first rose to public attention right near a raccoon-dog stall in a wet market. Either way is one of the century’s biggest coincidences, designed by some cosmic joker who wanted to keep the debate stayed acrimonious for years to come.
Scott messed up. The lab was built near wet markets because wet markets were a threat, and GOF research probably was conducted there because leaks could be blamed on wet markets.
The new notes USRTK obtained through FOIA in January 2024 aren't trivial. I had been pretty much on the fence about the lab leak hypothesis, but the new evidence is so overwhelmingly damning it literally made me nauseous reading it. The notes explicitly propose generating a consensus sequence with an inserted furin site. They propose using six BsmBI sites to assemble the infectious recombinant - confirming a 2023 hypotheses that the spacing of the six BsmBI sites in SARS-CoV-2 had a suspicious spacing. The notes proposed passaging the recombinant virus in human cell lines under BSL2 conditions.
It's a mistake to casually dismiss new smoking guns, all caps or otherwise.
"Why did the older strain start spreading later? Probably the virus crossed from bats into raccoon-dogs on some raccoon-dog farm out in the country. It spread in the raccoon-dogs for a while, racking up mutations, including the (less mutated) Lineage A strain and the (slightly more mutated) Lineage B strain. Then several raccoon-dogs were taken to Wuhan for sale, including one with Lineage A and another with Lineage B. The one with Lineage B passed its virus to humans earlier. Then 3-4 days later, the Lineage A one passed its virus to humans."
So apparently the zoonosis story requires *two separate* zoonotic transmissions, from the same location and within the same week, but from separate viral lineages? Am I understanding this right? And if so, then can someone please explain why both sides are just passing over this claim as a minor aside rather than a devastating hole in the theory?
I am not an expert in the field, at all, but this intuitively strikes me as *really* implausible.
Side note about the use of "conspiracy theory" in this context. It's got these two meanings, one being "nutty, unfounded belief" and another being "a theory about how people coordinated actions secretly behind the scenes." There's a Venn diagram where flat earthers believe an endless stream of government officials are conspiring to keep the truth from us.
It doesn't seem accurate to me to call the lab leak idea conspiracy theorizing in the "nutty, unfounded belief" sense. Smart people in very recent years, including in this debate, disagree. When you accuse the other side in that situation of "conspiracy theorizing", you're just engaging in ad hominem attacks which add no useful information.
Some lab leak people do believe that there was behind-the-scenes conspiring that happened, but so do some zoonotic supporters. There are more people on the lab leak side who do seem to be more of the nutty, unfounded type of believers. But it's a distraction to look at them, because the evidence in favor of lab leak is not like the evidence in favor of the earth being flat, etc.
Sometimes too the people we call conspiracy theorists are just people who have been slower to catch up to the preponderance of evidence, given that science is a process that unfolds over years. I don't think it's helpful to call people in this category conspiracy theorists.
Strong arguments were made on both sides of this. Peter's arguments look stronger. But as a person who doesn't have their hands on all the raw data, I wouldn't say that he "debunked" the other side. He just presented some other information that was more persuasive.
Anyway, I guess this is a wish for not calling people "conspiracy theorists" as it relates to things scientists themselves are still debating.
The whole Session 2 issue seems irrelevant to me. The simplest lab leak scenario is a virus being collected in the wild, brought to the lab, then escaping in guano on a tech's scrubs or something similar. No need for genetic engineering of the virus at all.
The argument seems to have been "natural virus in wet market" versus "engineered virus from lab" while neglecting other scenarios.
"As mentioned earlier, the DEFUSE grant was rejected."
I do not know how many people here are familiar with how the federal grant system (whether NIH or DoD) for biomedicine works, but if one claims that the only way experiments get done is if 1) they are proposed in a grant and 2) that grant is funded, they are either unfamiliar with how the system works (like Peter) or else engaging in intentional misdirection (like the many virologists who have made this claim and certainly know better). As a general rule, the proposed experiments in a biomedical grant fall (often roughly equally) into 4 categories: 1) work you have already done but not published and inot included as preliminary data (you have to include just the right amount of data to justify the proposed experiments/money) ; 2) work you never intend to actually do but which looks good on the application; 3) work you do during the 1-2 years the grant is under review but no decision has been rendered; 4) work you actually will end up doing during the funding period.
That the DEFUSE grant itself was not funded is drastically less important (frankly not important at all) from the perspective of a Bayesian trying to figure out the origin than the fact they proposed making the precise modification that is so striking about Covid and has not been seen in any close relatives. David Baltimore gets this, Nick Patterson gets this, and I think anyone with knowledge of both how these grants work as well as the molecular biology gets this.
Saar raised the issue of ascertainment bias, but undersold it. There's no question imo: the case search *was* heavily focused on HSM, and the HSM sample search *was* heavily focused on the southwest corner. The Chinese authorities who did the search have been clear on this (Bahry, 2023, Table 1 and Fig. 1).
E.g. we now know that the raccoon dog stall was sampled 10 times, with 5/10 positive; while the pool of blood outside a beef stall in the east wing was only sampled once, with 1/1 positive (Liu et al., 2023; cf. Bahry, 2023, Fig. 1). That, not the racoon dog stall, was the highest positivity ratio. Peter's heatmap was based on a paper by some Western scientists who had no idea how many times each stall had been sampled, and had instead speculated that every sampled stall had been sampled equally (Worobey et al., 2022).* After Jesse Bloom also showed there's no positive association between raccoon dog and SARS-CoV-2 genetic material, nothing implicates them, not even in a God's-joke way (Bloom, 2023).
China CDC director George Gao has also emphasized, when asked by the host of a BBC podcast, that this overwhelming ascertainment bias means you can't assume genuine case clustering around HSM (https://www.bbc.co.uk/sounds/play/m001ng7c, 24:00–25:12).
References
Bahry, D. (2023). Rational discourse on virology and pandemics. mBio 14: e00313-23. https://doi.org/10.1128/mbio.00313-23
Bloom, JD. (2023). Association between SARS-CoV-2 and metagenomic content of samples from the Huanan Seafood Market. Virus Evolution 9: vead050. https://doi.org/10.1093/ve/vead050
Liu, WJ. et al. (2023). Surveillance of SARS-CoV-2 at the Huanan Seafood Market. Nature. https://doi.org/10.1038/s41586-023-06043-2
Worobey, M. et al. (2022). The Huanan Seafood Wholesale Market in Wuhan was the early epicenter of the COVID-19 pandemic. Science 377: 951–959. https://doi.org/10.1126/science.abp8715
*This zoonosis-theory paper is one of many by a prominent coauthor network centered on "Proximal origin" authors Andersen, Holmes, Rambaut, and Garry.
Question for Peter: don't you think you're being a little credulous with respect to the early case map, given an obvious and known effort from WHO/China to support anything but LL?
Separately--great job. To go into this with nothing but your own research and bet a good portion of your net worth against some rich guy, and win, is something from a movie. You should be proud for sure.
What should we make of this study, which found the presence of covid in Brazilian wastewater in late 2019?
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7938741/
Well, all of this just confirmed my priors: most debate is utterly worthless. They can talk about likelihoods and logical fallacies all they want, but the problem is that no one has any hard evidence. Peter's obviously the smartest one in the room here, but just being intelligent is of no use if you don't have anything to work with. Sure, you can take a bunch of circumstantial evidence and then claim that they make your theory more likely by pulling numbers out of your ass, but none of that gets you any closer to verifying the actual truth. And at this point, actually verifying the cause of COVID is likely impossible-- any hard evidence would have been burned by the CCP, since even they don't have anything to gain by knowing the truth.
...So what's even the point of all of this? As others have pointed out, the actual dangers of wet markets or GoF research is completely independent of whether or not COVID in particular was caused by either of them. One event doesn't change the reality of the systems at play. We already agree on the information that matters: that the situation was potentially preventable by the CCP, and that they failed to do so. But even that wasn't enough for countries to stop doing buisness with them, so how would figuring out the origin of COVID change anything? This entire discussion is utterly pointless.
This is a highly interesting topic. Unfortunately I don't have the energy tonight to get deep into all the great points brought up in the post. But I want to share a few scattered thoughts in babble mode:
(1) Taking a 5% chance of dying on a climb is pure insanity. Maaaybe taking a 5% *lifetime* risk is worth it if you really love climbing. Hardly an endorsement for risking $100k at 4%.
(2) It's a terrible idea to offer a standing bet on your true beliefs. You're literally doing adverse selection and end up with a sneaky rules lawyer as your debate "partner".
(3) It seems that Peter is just a good debater, of the "debate club" variety. Laser focused on winning the debate, rather than finding truth. I was repulsed by the dirty arguments made, for example the quibbling about whether to call the first outbreak a "superspreader event" or not. Regardless of the completely irrelevant doubling times Peter brings up multiple times, a place like the Wuhan market clearly is a mechanism for turning a handful of cases into 80k cases. I've only skimmed the *summary* of the debate, so please point me to where Peter tackles the real underlying issue head on.
(4) Re: orders of magnitude. There is no limit to how badly you can fuck up a bayesian estimate. For example, Peter's 1 in 10'000 probability of the first infected person working at the wet market. That's just a uniform prior! It's so, so bad. The easiest way to debunk this is to take these odds, and start listing conclusions that would need to follow if they were actually correct: "viruses are equally likely to spread to everyone regardless of their profession and their position in the social graph"... um no, everyone with an inch of common sense (or kids) knows that viruses spread where people gather: schools, workplaces, and yes, markets. The vendor at that market didn't need to be the first, merely among the first *few*, and then keep coming to work while contagious.
(4a) It strikes me this might be a valuable method to flesh out. Let's call it the *reverse inference test". I don't know why, I just feel like it latches onto my intuitions more cleanly than trying to refute a claim in the same direction it's made.
(5) Probability cascades are bullshit. To me, this is the classic example of "science as attire"; you pretend to be scientific by multiplying numbers together, but what you're really doing is spinning a narrative. It's extremely hard to come up with an exhaustive enumeration of posibilities. MECE, as it's sometimes called (mutually exclusive, collectively encompassing). Or for the mathy people, an equivalence relation on the possible states of reality. If you doubt this, go take an advanced algorithms class and report back. I'm not joking... repeated encounters with impartial yet implacable entities (compilers, test suites, theorem provers, etc) will drill this into you like nothing else. Peter's argument style is that of a college freshman, and any competent math mentor would beat it out of him before it rots his brain. To the extent Peter is aware of this, it just reinforces (3).
(5a) to clarify the above point, the typical mistakes made is multiplying together probabilities that aren't independent, and to implicitly assume that the only way to reach state X is the specific path towards X you've laid out. These can make a 50% event seem like a 1% event, or the other way around.
(6) Explicit bayesian calculations in the absence of a *formal model* are a horrible idea. It just takes confusion and multiplies it by orders of magnitude. What I've learned in my time as a scientist is that the relationship between the real world and the model you're working on, and how to transport assumptions and conclusions back and forth between them, is one of the hardest things to get right. And we don't really have a systematic way of teaching this skill.
(7) If you wanted to blackpill me on the applicability of bayesian calculations to everyday life, you've succeeded. In particular, I see no reason to update my beliefs regarding the lab leak hypothesis in response to this debate. I'm still 50%+ in favor.
Wow, that was a lot more than I thought. Thanks for reading, and I hope it was helpful, even in this low-effort format.
I know I am being ridiculous, but has anyone made a 2-3000 word summary of this post? Would be very useful for semi-literates like me.
My brain skipped a beat when I read the summary that Peter dismissed one of the later market spreads as "that didn't come from a human source, there were COVID particles on the fish". That feels eerily lab leak mechanics adjacent and I found myself slightly confused why he thought that was likely but COVID from the lab coming to the wet market and then spreading from there unlikely. Probably if I watched the videos, I would be less confused about this, and I'm not trying to make an argument, just registering my confusion.
Generally, I have no strong position on this question at all and a low prior on that the things I think about it are true. I thought this article was a fantastic deep-dive! I realise I could get even more information if I watched the videos, but I'm not really a video person. Thank you so much for this. :)
Regarding the 1 in 10000 chance of being discovered in a wet market if lab leak:
That is flawed. You might as well assign 1 in 365 that it was discovered December 30th. You can tell it is flawed because if it had been discovered in a school, Saar would assign 1/10000. Or in a restaurant, 1/10000. Or a bakery, 1/10000. He gets a free 1/10000 wherever because any public place can be differentiated by at least 1/10000. “Only 1/10000 Wuhans work at Wuhan IHOPs!”
You nod at this briefly in the conclusion, but what I find fascinating about this debate is that the what truly happened doesn't seem to matter at all! If an oracle arrived and told us the source, I don't think it would (or should) change our planning at all. The investigation of COVID has revealed that there are huge risks from *both* lab leaks and zoonosis that we need to invest significant effort in reducing, and which one actually happened in this case doesn't change that.
Maybe I'm missing something, but it seems like Peter's argument hinges on two separate cases of zoonosis within days of each other in the exact same location and the same mutation? Or maybe I misunderstand that point.
Many Thanks!
>The six estimates span twenty-three orders of magnitude. Even if we remove Peter (who’s kind of trolling), the remaining estimates span a range of ~7 OOMs. And even if we remove Saar (limiting the analysis to neutral non-participants), we’re still left with a factor-of-50 difference.
ouch, ouch, _ouch_, _OUCH_.
And I thought that the single order of magnitude disagreement between the superforcasters and the domain experts on AI risk was bad... ( as I'd commented in https://www.astralcodexten.com/p/mantic-monday-31124/comment/51596120 )
In a way, the scatter in estimated here is even more disturbing, because AI risk unavoidably involves estimates of future events, many with only weak analogies to known past events. _This_ estimate is all about events which _do_ have strong analogs to known past events.
I don't know what to suggest. One large class of worries is indeed treating some probabilities as uncorrelated when one could indeed expect correlations. E.g. as you said, researchers modifying viruses in ways intended to be less researcher-like and more nature-like in order to anticipate natural changes.
One other thing that generally bothers me is the blanks in a lot of the probability calculations where one side estimated an effect and the other side just didn't discuss it. Maybe it would have been helpful to have a final pass where all sides gave estimates for _all_ of the factors?
Does peter have a blog for us to follow his future work?
So, I should go back to pitching my children's animated TV series "Wuhanimals" about a raccoon dog, a bat, and a pangolin who are friends?
I truly hate to be a meta-commentator refusing to engage with the meat and bones of the debate, but I'm still puzzled by why this matters enough for the debate to be worth at least 100000 dollars and a few dozen hours of a few intelligent people's time (probably a lot more time and money, actually, given the extent to which all of this was discussed in other sources, of varying levels of sophistication, that I don't follow).
Would it help anyone in particular, even people who lost loved ones or whose businesses did not survive the economic consequences of the pandemic, to know whether the virus was zoonotical or originated from a WIV leak?
I doubt there would be meaningful political or economic consequences either way. I'm moderately sure virological research is still being funded about the same way it did before the pandemic, I'm a bit more sure wet markets are still as unsanitary as they were before the pandemic, I'm quite sure trade/political relations/economic dependence of the world on Chinese economy and industry is about the same as it was before the pandemic, and I'm very sure human beings still die the same way they did before (and during) the pandemic.
Great write up, and incredibly intellectually stimulating. One thing that struck me is how much harder it is to understand the past than to predict the future! If Rootclaim is so good, it should prove itself on the prediction markets, instead of relying on judges. This also solves the problem of financial imbalance, challenging people to high stakes debates is like bullying someone at the poker table when you have a pile of chips.
Importantly, Miller incorrectly claimed the N501Y mutation would result from passage in hACE2 mice (mixed them up with BALB/c mice). The major papers Miller relied on have been seriously challenged since the debate. See Stoyan and Chiu (2024), Weissman (2024), Bloom (2023) and Lv et al (2024). Overall the circumstantial evidence makes lab v plausible:
1. Chinese researchers Botao & Lei Xiao observed lab origin was likely given the nearest known relatives to SARS-CoV-2 were far from Wuhan. Wuhan Institute of Virology (WIV) sampled SARS-related bat coronaviruses where the nearest relatives are found in Yunnan, Laos and Vietnam ~1500km away. They refuse to share their records.
2. Patrick Berche, DG at Institut Pasteur in Lille 2014-18, notes you would expect secondary outbreaks if it arose via the live animal trade. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10234839/
3.Molecular data: Only sarbecovirus with a furin cleavage site. Well adapted to human ACE2 cells. Low genetic diversity indicating a lack of prior circulation (Berche 2023). The CGG-CGG arginine codon usage is particularly unusual but used in synthetic biology.
Restriction site SARS-CoV-2 BsaI/BsmBI restriction map falls neatly within the ideal range for a reverse genetics system and used previously at WIV and UNC. Ngram analysis of the codon usage per Professor Louis Nemzer https://twitter.com/BiophysicsFL/status/1667232580255490053?t=IJgitS5cw364ioclzVWxaA&s=19
The SARS2 backbone is very low in CG and CpG. While the 12-nt insert that gives it the FCS is extremely high in both. Almost as if it was some kind of chimera of a consensus sequence and a codon-optimized polybasic cleavage site? https://twitter.com/BiophysicsFL/status/1752800486837678377?t=EpIRgyybJVaPgeMP5xdstA&s=19
https://www.biorxiv.org/content/10.1101/2022.10.18.512756v1
https://link.springer.com/article/10.1007/s10311-021-01211-0?fbclid=IwAR1HMUMtLIAFOFppVasQDeoIAYrVhP8j4YoPO4wnaTOUiKLsllZl_oKryOw
4. DEFUSE full proposal: virus 20% different from SARS1, consensus seq assembled with 6 segments, without disrupting coding seq, BsmBI order, FCS. SARS2: 20% different than SARS1, 6 evenly spaced fragments w BsmBI and BsaI restriction sites, FCS.
Jesse Bloom, Jack Nunberg, Robert Townley, Alexandre Hassanin have observed this workflow could have lead to SARS-CoV-2. Work often begins before funding sought or goes ahead anyway.
5. Market cases were all lineage B. Lv et al (2024) indicates there was a single point of emergence and A came before B. So market cases not the primary cases. See also Bloom (2021), Kumar et al (2022). Peter Ben Embarek said there were likely already thousands of cases in Wuhan in December 2019.https://t.co/50kFV9zSb6
https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/34398234/
https://academic.oup.com/bioinformatics/article/38/10/2719/6553661
6. Evidence for lineage A in the market is based on a low quality sample according to Liu et. al. (2023).
7. Bloom (2023) shows market samples do not support market origin. There is also no evidence of transmission in the claimed susceptible animals elsewhere. https://academic.oup.com/ve/advance-article/doi/10.1093/ve/vead089/7504441
8. Lineage A and B only two mutations apart. François Ballox, Bloom and Virginie Courtier-Orgogozo note this is unlikely to reflect two separate animal spillovers as opposed to incomplete case ascertainment of human to human transmission (Bloom 2021).
9. Sampling bias. George Gao, Chinese CDC head at the time, acknowledged to the BBC stating they may have focused too much on and around the market and missed cases on the other side of the city. David Bahry outlines the documented bias. Michael Weissman has shown this mathematically.
https://journals.asm.org/doi/10.1128/mbio.00313-23
https://academic.oup.com/jrsssa/advance-article-abstract/doi/10.1093/jrsssa/qnae021/7632556
10. Spatial statistics experts show the Worobey claim the market was the early epicentre was flawed.
https://academic.oup.com/jrsssa/advance-article-abstract/doi/10.1093/jrsssa/qnad139/7557954
11. Wuhan used as a control for a 2015 serological study on SARS-related bat coronaviruses due to its urban location.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6178078/
12. Superspreader events also seen at wet markets in Beijing and Singapore (Xinfadi and Jurong).
13. WIV refuse to share their records with NIH who terminated subaward in 2022. Wider suspension over biosafety concerns. https://www.bloomberg.com/news/articles/2023-07-18/us-suspends-wuhan-institute-funds-over-covid-stonewalling
14. PLA involvement at WIV and MERS research prior to SARS-COV-2. MERS features several similarities with SARS-CoV-2.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7022351/
15. SARS1 leaked several times and SARS-COV-2 has leaked from a BSL-3 lab in Taiwan.
16. Unpublished infectious clone identified from Wuhan contradicting arguments such reverse genetics systems would be published.
https://www.biorxiv.org/content/10.1101/2023.02.12.528210v1.full
Miller incorrectly claimed the N501Y mutation would result from passage in hACE2 mice (he mixed them up with BALB/c mice). Since the debate the key papers that Miller relied on have been badly undermined:
1. Worobey et al featured on Retraction Watch after spatial statistics experts eviscerated their claim the Huanan Seafood Market was early epicenter. https://academic.oup.com/jrsssa/advance-article-abstract/doi/10.1093/jrsssa/qnad139/7557954
2. Lv et al (2024) found new intermediate genomes so the multiple spillover theory is unlikely (it was anyway given lineage A and B are only two mutations apart). Single point of emergence is more likely with lineage A coming first. So market cases are not the primary cases (all lineage B). Their findings are consistent with Caraballo-Ortiz (2022), Bloom (2021).
t.co/50kFV9zSb6
3. Jesse Bloom showed again the market samples don't support market origin. t.co/rorquFs1wm
4. Michael Weissman shows mathematically ascertainment bias in the early case data the judges relied on (George Gao, Chinese CDC head at the time, acknowledged this to the BBC last year - they focused too much on and around the market and may have missed cases on the other side of the city).
https://academic.oup.com/jrsssa/advance-article-abstract/doi/10.1093/jrsssa/qnae021/7632556
5. Account that identified errors in Pekar et. al. leading to an erratum last year has found another significant error. Single spillover looks more likely. t.co/GAPihZu51P
> I have a weird urge to visit Wuhan as a tourist, see the Wuhan Institute of Virology, stroll through the Huanan Central Seafood Market (unfortunately closed), maybe eat a raccoon-dog.
Scott N Alexander N plans N to N eat N Raphtalia N CONFIRMED
Thank you for an incredible write-up and analysis.
I am actually shocked at how poorly Saar fares here. The vast majority of his evidence is basically "if things were different in a way we can't know, then our conclusions would also be different, therefore lab leak." E.g. "if COVID spread rate was lower in the beginning", "if an infected WIV worker spread to the wet market then isolated", and so on. That's it, really? I was expecting so much more actual evidence. And then what data points they do provide (e.g. "Mr Chen", 90 cases, intermediates, etc) have apparently valid reasons why they're not evidence. I started out 60/40 zoonotic, but based on this I feel I have no choice but to update to 99/1.
I feel the real problem with this setup/framework is that a specific position is taken, with overly-precise numbers, and the goal of the debate is to simply justify that final percentage. You see this with Saar's "but the judges just did the math wrong" sour grapes complaint.
In the IT and infosec world, we have the FAIR method--a similar method for building quantified models of risks. The real value of these isn't that you convinced yourself you have only a 1-in-10000 chance of being compromised by a state actor in the next 10 years... it's the shared understanding of the risks your business thinks are important, how you think those risks are correlated, how you think they're evolving over time, etc. That enables your organization to act from shared goals, and not get bogged down in interminable debates day-to-day. The final numbers are fine, especially if you use confidence intervals, but you know it's only a snapshot in time using your best data and estimates as of the last time you underwent the exercise; by the time you update in a year/quarter, you will have more data and different estimates. Sometimes, risks that weren't even on your model last cycle, are now the top priority because of what's happening in the world (or because you gained new information that you missed previously). It's a fact of life.
With that in mind, I think Saar fares better if his arguments are framed as reducing absolute confidence in zootonic origin, and not the reason why LL is the only possible explanation (which it obviously isn't, even from Saar's own arguments). So yeah, I was only joking about updating to 99% in favor of Z--I think there's enough doubt thrown in from Saar's evidence alone that it could not possibly be more than 80%. But Peter/Z still wins this version of the debate, and Saar's over-confidence in "no way it's anything other than LL, here's the math to prove it, and if you disagree then you're doing the math wrong" does border on pseudoscience here.
To that end, I think Scott's final takeaways are the most important. If we have any shared goals anymore as a society, then we should devote resources to better preparing for pandemics from both origins, and in also improving government/organizational transparency so it's easier next time to understand what went wrong. As it is, either a natural disaster can be ignored because "teh government conspiracy," or else somebody got away with murder; either way, the debates that focus solely on convincing everybody that one's favored position is the only explanation means we're not more prepared for future disasters than we were with this one, we're just more divided and easier to manipulate.
I found this debate to be very valuable. I started with 70:30 odds on lab leak's side (I didn't do any extensive research on COVID origins) but after this debate that shifted to 5:95 (i.e., favoring zoonosis). I wish there are more such debates on contentious topics.
The Rootclaim model seems pretty sensitive to modest differences in probabilities; I'm guessing a factor of having too many variables so small differences across all of them add up to a big effect? I tried to average the six guesses giving Saar the least weight and add some of my own intuition and got (I think) a modest pro-lab leak outcome. https://docs.google.com/spreadsheets/d/14OkrvJ21qCqQcXINQnMO4-a1Xi-npidF/edit#gid=2057332411
> About half of cancer patients lose their hair, and Putin hasn’t, so we’ll divide by two.
This leapt out as methodologically unsound. How do you know whether Putin's lost his hair? Here are some possible scenarios:
1. Putin has never had cancer or anything cancer-like. His hair is fine.
2. Putin has had chemotherapy. He wears, or has worn, prosthetic hair to hide this fact.
Note that these two scenarios are perceptually identical. You have no way to tell one from the other.
If the question you're evaluating is "Is Putin covering up a case of cancer?", his visible hair provides no information at all, because all scenarios predict visible hair. In the "no" case, the hair is irrelevant, and in the "yes" case, the hair is deceptive.
It's kind of like making an adjustment to your evaluation of "Is this a coral snake, or is it a king snake?" for the observation that the snake in question has red and yellow stripes. That observation was already part of the premise of the question; you're double counting (among other weird mistakes) if you adjust for it again.
> Another of Saar’s concerns with the verdict was that Peter was an extraordinary debater, to the point where it could have overwhelmed the signal from the evidence.
I've mentioned this before in the ACX comments, but this is such a great lead-in that I'll say again that this is why I didn't make a habit of reading Overcoming Bias. Eliezer Yudkowsky used that blog to make a lot of excellent points in an entertaining and well-written fashion.
And I got the sense, reading his work, that sounding persuasive was a bigger part of his strategy than being right, meaning that reading him was dangerous. So I enjoyed a number of pieces, but I didn't follow him or seek out his work.
My suspicion here is that using estimated probabilities instead of probability distributions leads to the same issue as for the Fermi paradox, in that your result is extremely sensitive to your assumptions and you end up with extreme outcomes on either side like we have here: 10^-25 and 534 are both clearly wrong.
My overwhelming feeling while reading this was a sense that a lot of the important evidence is itself subject to long debates and explanations that are being glossed over. Is a certain length of protein sequence likely or unlikely to be done by a mutation in the wild? What is the right way to construct priors about wet market and virus lab locations? Everyone was just stating certain things confidently, while providing detailed documentation for other things.
In the end, “evidence for my claims” seems to have a fractal quality sometimes. The closer you look, the more evidence you need to back up your use of evidence. It’s turtles all the way down.
While I'm terribly sympathetic to most of Wilf's points, one thing I absolutely agree with him on is the superiority of written to real-time/spoken debate, because the things it rewards correlate much more strongly with being right.
It seems to me that Saar's evidence-weighing analysis is basically "outside view" with extra steps.
As I see it, Saar's analysis hinges on the point that the zoonosis hypothesis contains a coincidence that is very unlikely to be explained by some out-of-model error (which I agree with), whereas the lab leak hypothesis only contains a coincidence that could very plausibly be an out-of-model error (I agree with this as well). This sets up a barrier that is almost impossible for any amount of mere digging into the details to overcome, as that details-digging is all itself quite vulnerable to out-of-model errors. And so the end result is an outside-view judgement that no amount of inside-view analysis can outweigh.
This surely seems like an error to me, though those with different positions on the outside-versus-inside-view debate might disagree.
I genuinely do not understand why there is not more talk about the third option: that one of the many bat cave spelunkers from Wuhan caught COVID-19 and brought it back to Wuhan.
The WIV and the market were fairly far apart from each other, as noted in the debate. The Wuhan CDC is *right next to the market*. And while it doesn't seem that the Wuhan CDC was conducting gain of function research, their employees were fanning out all over China going into bat caves.
From this Washington Post article - https://www.washingtonpost.com/world/asia_pacific/coronavirus-bats-china-wuhan/2021/06/02/772ef984-beb2-11eb-922a-c40c9774bc48_story.html
"In the video, the researchers scale the cavern wall, their headlamps ghostly blue.
“If our skin is exposed, it can easily come in contact with bat excrement and contaminated matter, which means this is quite risky,” says Tian Junhua ["associate chief technician in the Wuhan CDC’s pest-control department, but he has a reputation as a swaggering adventurer in his work with bats and insects"], one of the bat hunters.
...
The video was released by national science authorities and Chinese state broadcaster CCTV on Dec. 10, 2019, and circulated on social media. It’s a high-quality production, designed to promote China’s world-leading viral research.
...
Tian and his team from the Wuhan Center for Disease Control and Prevention are filmed catching horseshoe and pipistrelle bats and collecting samples of guano, in search of new bat-borne diseases and the basis of new vaccines. Tian talks about the need for caution. “It is while discovering new viruses that we are most at risk of infection,” he says, though he is shown handling sample vials without wearing full protective gear.
...
In 2017, Tian told the state-run Wuhan Evening News he once forgot personal protective equipment and was splattered with bat urine, leading him to quarantine at home for two weeks. On multiple occasions, bat blood squirted onto his skin while he was trying to grasp the animals with a clamp, he told the paper."
So the questions would be - exactly when did the Wuhan CDC move to be close by the market? The move finished on December 2, 2019 - how long was the move and how was it conducted? Does the science require an intermediate animal or could it have jumped directly from bats to humans with the cave exploration?
I am now scared that the next pandemic will be started by debaters on either side doing gain of function research to shore up the evidence for auxiliary points in their debate.
There's so much stuff on this page that Chrome is having difficulty rendering it even on a high-end desktop. Consider closing comments and creating a separate page for that.
At the outset of the pandemic, I was at about 15% lab leak, 85% natural origin, and thought it unimportant because I didn't think we'd ever have enough evidence to do better than guess. By the start of this year, after way more than 15 hours of studying way more evidence than I imagined we would ever have, I was at 80% lab leak, 20% natural origin, After this, maybe I should update to 70% lab leak, 30% natural origin. But I'm going to indulge in a bit of emotion and make that 25% lab leak, 100% annoyed.
Annoyed in that it seems like the rationalist and rationalist-adjacent community, parts of which at least I value and respect, has decided that on this one issue we're going to flip a $100K coin to decide what to believe, and I suspect anyone who dissents will at best be quietly dismissed as an ignoramus by much of the community. I will grant that by paying $100K, we got a coin that is *slightly* biased towards the truth,
Seriously, one-on-one personal realtime debate is a *lousy* way to find the truth. There's a reason we don't have debates as a central feature of e.g. scientific conferences. Throwing in high-value bets doesn't improve things. The process selects for the most skilled and charismatic debater, and in some cases for the favor of biased judges, with the truth mostly just serving as a tiebreaker.
In this case, there was no tie to break. Miller, by all accounts including this one, is an exceptionally capable debater. Wilf, while clearly skilled as an entrepreneur and possibly as an analyst, does not seem to be very good at in-person debate. From this very thorough account, Wilf completely failed to raise some of the very strong points in favor of the lab leak hypothesis, failed to call out Miller on some of his indefensibly weak points, and agreed to a structure where he'd open by spending the first third of the debate on the very weak assertion, "maybe it didn't come from the Wet Market" even though that is in no way necessary to the lab leak case. Or maybe he did raise the strong points and call out Miller's weak points, but in a way that neither Scott nor the judges noticed.
As for Scott shifting is odds from 50-50 to 90-10 on the basis of this debate, I think that requires p<0.20 for "Wilf isn't very good at this and left the winning arguments at home", which I don't think is defensible. You'd need a statistically significant sample of debates to be confident you have seen both sides well-argued.
I will continue to value and respect the rationalist and rationalist-adjacent community, particularly Scott's corner. But it has a clear blind spot in its enthusiasm for clever and novel truth-seeking mechanisms (bets, prediction markets, debates, etc) on the basis of their cleverness and novelty without adequate regard for their in many cases obvious flaws.
> The paper that claimed that defined how well COVID was adapted
Typo
If it turns out it was natural origin — as I am more likely to now believe — it’s sad it will be that much harder to end gain of function research. Even if the lab did not leak the pandemic, the risk that the lab would leak a pandemic eventually was still too damn high. Hopefully this event will nonetheless lead people to take the risks more seriously.
I’ve spent some time in virologist subreddits since covid, and I find their attitude towards these risks to be terrifying. They remain very blasé about it. It appears to be deep in their culture, similar to how civil engineers are culturally committed to designing for cars, and architects are culturally against ornamentation.
The science has to decide whether it’s a lab created virus or not. However created in a lab is not the same as lab leak. In fact your prior for this being a leak should be close to, or at, 0% since it’s the only time a major pandemic has happened in that fashion.
When you look at the distance between the lab and the market and the fact that the origin is clearly the market if lab created pathogen is proven true then the most likely explanation is an external attack.
> So for example, suppose there are fifty things about a virus. You should expect at least one of those to have a one-in-fifty coincidence by pure chance.
Minor math error here I think: You should expect *exactly* one, not at least one. It's equally likely to be more than one or less than one.
Thank you to Saar for running this in good faith, and thank you to Scott for a great write up. One of the key things that stood out to me while reading the analysis was that there are a few relatively simple factual items that were disagreed on that might be relatively easy to chase down if we weren't worried about governments trying to obfuscate things. There's also a few simple pieces of information that could shift my thinking a bit too, like where do the lab workers live? I wouldn't expect a lab leak outbreak to be centered around the lab, but around one of its employees homes.
I noticed in myself a high risk of falling victim to a classic 'david and goliath' story too, which is potentially a very bad bias. Almost everyone reading that first section is going to be rooting for Peter, and enjoy it everytime he seems to win a point.
I must admit that I don't care nearly as much about whether COVID was zoonotic or not, as I do about the government paying social media sites to enforce a particular orthodoxy about the question.
Some of this sounds like an “Encyclopedia Brown” level of argument. E.g. the idea that because the University of North Carolina said they’d do furin site insertion in a grant application from years before the pandemic, therefore the WIV cannot have done any insertions of their own.
Research institution capabilities, and researcher ambitions, and the real world generally, are a lot more complicated than that reasoning implies (I say this despite putting little weight personally on COVID being engineered).
Likewise with the idea that the WIV would have fully released the details of all viruses they have sequenced.
I haven’t watched the debate though, so maybe it was better than Scott makes it sound.
It strikes me that whether you should debate theories you think are crazy depends on how popular they are. Consider:
1/10000 believe in theory A. You debate someone arguing A, and you win 99% of the audience (considering what humans are like, this is super impressive). But now, 1/100 of the audience believes in the crazy. And *no-one* can convince 99.99%.
66% believe in theory B. Now when you debate someone arguing theory B, even if you only convince *half* the audience, things have improved!
This, of course, assumes that the people believing what you think is crazy can be convinced in the first place. This might well be true about lab leaks or *possibly* even UFOs; it will *not* be true about creationism (evidence isn't even a factor in why someone is a creationist).
The JFK assassination is another of those cosmic jokes. Sensible people should not believe in conspiracy theories about it. But dear Lord, was Lee Harvey Oswald a weirdo, the rare person who had been involved with *all* the kinds of people who _might_ be suspected of conspiracies; and murdered before he had a chance to testify; and with an investigation that couldn't have seemed more suspicious and cover-up:y if it tried.
Really fantastic read. I already leaned towards zoonosis but this removed most of the remaining doubt. It so valuable to see one side put up their strongest arguments and then see if the other side can knock them down, which in this case they did pretty dramatically.
Absolutely fantastic.
“… And so on and so forth, until we end up with the final calculation: 86% chance Putin doesn’t have cancer, too bad.”
What Rootclaim is doing is interesting. But should not such hard-core Bayesianism also include sensitivity analyses? That is, investigate how robust the conclusion is (here: “86 percent chance”), at least for minor-to-moderate changes in priors, age-related cancer statistics, and the like?
Using sensitivity tests in cost-benefit analyses is usually recommended. It appears that Bayesianism could benefit from such tests, for the same reason.
It would also quell the skepticism people like me automatically feel when encountering predictions spelled out in fine-grained percentages. (Plus, why stop at percentages? Why not add a few decimal places as well: 86,xy percent chance Putin doesn’t have cancer– that would make the prediction even more “accurate”….)
Do not get me wrong – I respect what Rootclaim is doing. But I would be (genuinely) interested in why Rootclaim does not add/construct some kind of confidence interval around their percentage-estimates. (And why Bayesians in general do not do this.)
SARS2 leaked from a Wuhan lab but it's not Chinese junk
https://jimhaslam.substack.com/p/lets-be-blunt-sars2-leaked-from-a
This post caused me to strongly update against Bayesian methods. Maybe it's the impossibility of seeing accurate priors, or the tendency to engage in cherry-picked absurd probability estimates. Or maybe it's the qualitative difference between statistical probabilities prospectively estimated vs retrospectively measured. I no longer have confidence this method produces more signal than noise.
For example, what prior do you set for an outbreak in WSM? You might think that's low, but given multiplicity you're not really looking at a prior probability anymore, are you? Both a spillover event and a lab leak might have come from any number of places. We'd tell some plausible-sounding story about why that place was unique, then set absurd priors as a fence around that story. But at heart it's still a post-hoc story, and all the prior-setting in the world will only serve to reinforce your story with more plausible-sounding stories. This seems to be what Saar is fighting against in making adjustments for out of model variance, but that seems like the wrong approach.
The better approach is probably abandoning Bayes - at least explicitly/strictly. I'm not yet convinced about the whole Bayesian enterprise, but it feels like it's on much more shaky ground for me today than it ever has since I first learned about it.
I personally agree it was not a lab leak but a pretty important was lost in the debate (or at least poorly factored in). Namely China was making hiding evidence. While this may impact priors…the bigger impact is that, if it was a lab leak we only know what information was released (which almost certainly would be anything that boosted their preferred narrative) and do nit have all the evidence that was presumably withheld (which would be all the evidence they could suppress that went against the preferred narrative).
It’s kind of like sample bias in that the evidence you use to make the assessment is systematically biased.
I unsure how you assess these probabilities because they are fundamentally unknowable.
"maybe you should lower your disbelief in each hypothesis to something more reasonable, like 99%. But now the chance that the sun rises tomorrow is 0.99^100, aka 36%. Seems bad."
All these hypotheses are very correlated; extreme probabilities of base events are not required, just extreme probabilities of conditionals.
'If all 99 other hypotheses are false, and I'm not making some silly maths mistake or whatever, then hypothesis 100 is 99.999999% false' is not necessarily overconfident even for a human.
Right?
I have done my own extensive research and believe the evidence is compelling that there was a lab leak. IMO, Peter has a real axe to grind and his strong personal conviction against the lab leak hypothesis has made him blind to common sense.
The odds that the virus evolved free of human-engineering are probably much less than in 1/10,000. Peter has no idea how to figure those odds and frankly no one else does either. The actions of the Chinese government in stonewalling are far more convincing to me that there was a lab leak. If the Chinese government didn't have something to hide why be so opaque???
A few thoughts:
For an adversarial setup (such as debates or US court cases) to converge on something close to the truth, the case for each side needs to be represented at a much more similar level than was done here. Even if an innocent man is accused of a crime (so he *knows* that the truth will set him free), it is still generally recommended that they hire a competent lawyer to represent their side, as opposed to defending themselves. This is probably not as important in countries with a less adversarial court system.
Calling Peter a better debater doesn't quite capture what happened here. I don't think there was a great difference in rhetorical skill, but there was an extreme difference in both the level of preparedness and the commitment to the cause. This case involves 50+ pieces of circumstantial evidence of various degrees of strength, some favoring lab leak and some favoring zoonosis. All the ones favoring lab leak were heavily contested, but at least to me it did not seem like the same was true for many of the ones favoring zoonosis. This was probably a mixture of a few things. Saar didn't seem to not know enough of the precise details to effectively push back against some of the bolder zoonosis arguments, such as how confident we should be that we actually have a good picture of the first cluster. In other words, he conceded far too much, allowing the zoonosis side to look much stronger than I suspect many people still believe it is. Secondly, I don't think Saar fully understood what his "role" in this debate was. Instead of being a soldier for his side, he mostly stuck with his 2-3 year old argument for what he thought the "truth" was. Peter, on the other hand, seemed eager to actually win the debate, so he made the maximally strong argument for his cause, not granting anything, as opposed to trying to converge on the "truth". So for every piece of evidence, he was able to put on the "maximally pro zoonosis" spin, perhaps resulting in his absurdly low final lab leak probability.
It is perhaps interesting to think about how this compares to adversarial collaborations, which are popular in our community. Using the language of Julia Galef, one can perhaps think of an adversarial collaboration as a "dance" between two scouts. Similarly, a debate can perhaps be thought of as a battle between two soldiers. The incentive is to win, not to find the truth. The rootclaim debate looked like a battle between a soldier and a scout, with the predictable result.
(Disclosure that I support in BiosafetyNow, but these are my views and don't necesarily reflect those of BN)
The Twitter origins debate, which Scott understandably shies away from (ref. his most recent tweets, and ofc I certainly don't blame him) represented the richest, most boisterous most tumultuous, most rancorous, most acrimonious form of the origins debate I've ever encountered online. The current ACX substack post & its comments don't reflect the Twitter debate's diversity of opinions, evidence, and individual credibilities IMO.
The key items I feel are insufficiently highlighted (even if they are mentoned at least once) from my scan of the comments section:
• the value of 'adverse inference' facing:
° PRC's gov
° the most outspoken virologists (+ 1 evolutionary biologist, + 1 disease ecologist, and yes not *all* virologists) especially when relevant materials turned up from their side only following subpoena
• No progenitor or 'sister' virus with an MRCA dated to shortly before outbreak has been found in animal populations despite extensive sampling.
• I don't feel any post-outbreak (and some near-pre-outbreak) data out of the PRC should be accepted on faith; rather it & all conclusions drawn therefrom should be placed in its own little suspect epistemic box. By contrast, any data in non-PRC hands pre-outbreak is 'gold'. The remaining quadrant, data & analysis produced in the ordinary course ex-PRC but post-outbreak, is tainted by the need to avoid 'rocking the geopolitical boat'. Often we can't reference such data that for example underlies post-outbreak intel findings, like that of the ODNI (atop DoE, FBI, CIA DIA...) so I can only give it the 'silver' label treatment.
And transcending all these quadrants, verified data that turned up ex-PRC from leaks, FOIAs, sequences reconstructed from SDAs, etc. is 'platinum'.
1. Peer reviewed papers since the debate that show ascertainment bias in early case data and intermediate genomes that undermine the multiple spillover theory indicating lineage A arose first so the market cases are not the primary cases (Weissman 2024, Lv et al (2024), Stoyan and Chiu (2024).
2. Miller's claims about the furin cleavage site are dubious:
Miller incorrectly claimed the N501Y mutation would result from passage in hACE2 mice (he mixed them up with BALB/c mice). WIV was performing in vivo experiments in transgenic (human ACE2 expressing) mice and civets in 2018 and 2019 in SARS-like CoVs.
I think both sides may have also overlooked the MERS furin cleavage site shares several structural and functional similarities. The sequence looks quite similar. In 2019 WIV researchers were involved in MERS research. Yusen Zhou who died in mysterious circumstances in May 2020 and was involved in an early Covid-19 vaccine was also involved in this research that involved manipulating the FCS. t.co/7zcSUPR60T
Broad Institute biologist Alina Chan also observes the S1/S2 FCS PRRA insertion in SARS-CoV-2 generates a Class IIS restriction enzyme site (BsaXI). This was used by WIV and Ralph Baric at UNC previously.
Dr Andreas Martin Lisewski discusses similarities with a MERS infectious clone described in 2017 here. The argument no one would engineer an FCS like this overlooks the MERS example.
t.co/fAVUlJu0TK
I think it's wrong to characterize this as a tragedy for Saar and Rootclaim. Sure the contest didn't validate their methods -- but I think their purpose was bigger than just winning internet arguments. Their goal is basically furthering scientific knowledge. Every model and method is flawed, but some are still useful. I see this as a heroic story, and Saar's stubbornness doesn't negate his accomplishment. By gracefully hosting and funding this exercise, he's made us all better off and advanced our knowledge of important issues. And that is the most important thing.
So the Chinese want us to think that they caused a global pandemic?
Although not infallible, humans are actually pretty good at ascertaining the intentions of enemies and potential enemies, and that is why it is a universal human inclination. Being oblivious to the likely motivations and intentions of other people is a big evolutionary disadvantage.
> Newtonian mechanics wasn’t pseudoscience when Newton discovered it, but if someone argued for it today (against relativity), that would be pseudoscientific.
I disagree strongly with the ontological assumptions here. Newtonian mechanics and relativity are both great theories. Newtonian mechanics is superior for terrestrial, human-scale phenomena because it provides a simpler representation of the physical laws. Relativity is superior at speeds approaching c because the complexity is justified by the accuracy gained. To claim that Einstein > Newton for modeling the trajectory of a baseball would be the same error as saying that Newton > Einstein for modeling a black hole: both arguments would be domain-of-applicability errors.
I self-identify as a flat earther, because for 99% of daily life, the flat earth theory (viz., conceptualizing the Earth as a flat plane) is simpler and more useful than conceptualizing it as a globe. The error made by "actual" flat Earthers is not that their theory is bad - they have an excellent and very useful theory - it's that they misunderstand their theory's domain of applicability. Flat Earth theory gets you to the grocery store; global earth theory gets you to the moon.
Quoting a comment I made on a lesswrong:
I think most points here are good points to make, but I also think it's useful as a general caution against this type of exercise being used as an argument at all! So I'd obviously caution against anyone taking your response itself as a reasonable attempt at an estimate of the "correct" Bayes factors, because this is all very bad epistemic practice! Public explanations and arguments are social claims, and usually contain heavily filtered evidence (even if unconsciously). Don't do this in public.
That is, this type of informal Bayesian estimate is useful as part of a ritual for changing your own mind, when done carefully. That requires a significant degree of self-composure, a willingness to change one's mind, and a high degree of justified confidence n your own mastery of unbiased reasoning.
Here, though, it is presented as an argument, which is not how any of this should work. And in this case, it was written by someone who already had a strong view of what the outcome should be, repeated publicly frequently, which makes it doubly hard to accept the implicit necessary claim that it was performed starting from an unbiased point at face value! At the very least, we need strong evidence that it was not an exercise in motivated reasoning, that the bottom line wasn't written before the evaluation started - which statement is completely missing, though to be fair, it would be unbelievable if it had been stated.
https://www.lesswrong.com/posts/NrKQGyggC7jcersuJ/on-coincidences-and-bayesian-reasoning-as-applied-to-the?commentId=HnxnDuqwucFAsG6bP
Counterposition here- given the verified attempts of multiple authorities to shape the "narrative" around covid, plus the powerful incentives for them to do so, does this mean that the sum of evidence available to assess the situation is probably unreliable and unfit for the purpose of this debate? The added layer of information control typical of the Chinese government makes the situation even more opaque.
Maybe this isn't a debate which can be settled based on the available information. Maybe that is by design.
The real question for me is this- why is any of this important in the long run? The obvious answer is that it has some bearing on how the world might respond to future pandemics. If it is yet another zoonoses then all the efforts to date to prevent them didn't prevent this one and probably wont prevent the next one, so maybe we should attempt to apply democratic pressure to make governments do more. I'm not sure they can do much more either way.
If laboratory research is behind covid, then this is also not completely unprecedented (though the level of genetic engineering might be somewhat new). Again, the technology isnt something that can be easily regulated by governments. Nucleotide sequence synthesis is relatively centrally controlled by a few large companies with the technical capacity, but that is only one of many ways to generate genetically modified pathogens. Again, we could attempt to apply democratic pressure to democratic governments, but again that would likely have limited impact.
The only thing that really shines through with the whole covid affair was the government and health authority overreach, panic, incompetence and opportunism. Once again none of this is new. The corporate capture of the medical system has been progressing for decades. At best the whole covid affair is just another reminder that authorities do not often have the interests of the individual as their main priority, and even when they do their capacity to deliver functional outcomes is usually highly limited. All trends point to this situation worsening in the future.
Novel pathogens will likely always remain potential black swans, which means that most of the time authorities will over react to their emergence relative to the actual risk posed. Over time this will probably build up broad social resistance to government responses until a genuinely threatening pandemic emerges. The relative difference in perceived threat between pathogens that are novel and fast moving versus those that are familiar and slow (such as tuberculosis) is also worth contemplating. Our information ecosystem thrives on novelty. Nobody is going to buy a newspaper that reports the same steady rate of slow deaths from tuberculosis or malaria. No government minister is going to win the fanatical devotion of the masses by heroically fighting such unexciting killers.
>"Peter seems to have a photographic memory for every detail of every study he’s ever read. He has some kind of 3D model in his brain of Wuhan, the wet market, and how all of its ventilation ducts and drains interacted with each other."
Mostly I think it's just keeping up with the Standard Arguments™️, from papers and preprints by zoo crew (Holmes, Andersen, Garry, Rambaut, Worobey, Pekar, Rasmussen, Débarre, Crits-Christoph, Neil etc.—the "Proximal origin" authors and their collaborators). They're the group of Western virologists who ~yearly come out with a new paper finally proving market zoonosis for real this time (Andersen et al. 2020, Holmes et al. 2021, Worobey et al. 2022, Pekar et al. 2022, Crits-Christoph et al. 2023a, Crits-Christoph et al. 2023b, etc.) and get a ~yearly media circus about it. Maybe he also read some pop-sci books like Chan & Ridley's "Viral" for pro-lab (2021) and Quammen's "Breathless" for pro-zoo (2023). Probably also the Chinese HSM-sampling paper (Liu et al., 2023) and WHO-China joint study (2021), following zoo crew in how to interpret them.
E.g. Peter's heat map slide is just Fig. 1 of (Crits-Christoph et al., 2023a), which is an elaboration of Fig. 4 of (Worobey et al., 2022); and his "drains" argument is from Fig. 2C of (Crits-Christoph et al., 2023b).*
References
Andersen, KG. et al. (2020). The proximal origin of SARS-CoV-2. Nature Medicine 26: 450–452. https://doi.org/10.1038/s41591-020-0820-9
Bahry, D. (2023). Rational discourse on virology and pandemics. mBio 14: e0031323. https://doi.org/10.1128/mbio.00313-23
Bloom, JD. (2023). Association between SARS-CoV-2 and metagenomic content of samples from the Huanan Seafood Market. Virus Evolution 9: vead050. https://doi.org/10.1093/ve/vead050
Chan, A. and Ridley, M. (2021). Viral: The Search for the Origin of COVID-19. HarperCollins.
Crits-Christoph, A. et al. (2023a). Genetic evidence of susceptible wildlife in SARS-CoV-2 positive samples at the Huanan Wholesale Seafood Market, Wuhan: Analysis and interpretation of data released by the Chinese Center for Disease Control [preprint]. Zenodo. https://doi.org/10.5281/zenodo.7754299
Crits-Christoph, A. et al. (2023b). Genetic tracing of market wildlife and viruses at the epicenter of the COVID-19 pandemic [preprint]. bioRxiv. https://doi.org/10.1101/2023.09.13.557637
Holmes, E. et al. (2021). The origins of SARS-CoV-2: A critical review. Cell 184: 4848-4856. https://doi.org/10.1016/j.cell.2021.08.017
Liu, WJ. et al. (2023). Surveillance of SARS-CoV-2 at the Huanan Seafood Market. Nature [online ahead of print]. https://doi.org/10.1038/s41586-023-06043-2
Pekar, J. et al. (2022). The molecular epidemiology of multiple zoonotic origins of SARS-CoV-2. Science 377: 960-966. https://doi.org/10.1126/science.abp8337
Quammen, D. (2023). Breathless: The Scientific Race to Defeat a Deadly Virus. Simon & Schuster.
WHO (2021). WHO-convened global study of origins of SARS-CoV-2: China Part. https://www.who.int/publications/i/item/who-convened-global-study-of-origins-of-sars-cov-2-china-part
Worobey, M. et al. (2022). The Huanan Seafood Wholesale Market in Wuhan was the early epicenter of the COVID-19 pandemic. Science 377: 951-959. https://doi.org/10.1126/science.abp8715
*The heat map is misleading: it was based on implausibly assuming all sampled stalls were sampled equally; in fact the wildlife-corner stalls were sampled far more heavily (Liu et al., 2023). For a critique of dismissing ascertainment bias, see my (Bahry, 2023). For a critique of Crits-Christoph et al. 2023a, see Bloom (2023). For a critique of their later, also-misleading heatmap (Crits-Christoph et al. 2023b Fig. 2a), see (https://www.biorxiv.org/content/10.1101/2023.09.13.557637v1#comment-6279754825).
I find the "drains" argument mildly interesting (Crits-Christoph et al., 2023b Fig. 2c): nonzero evidence, but just one real datapoint. (The two downstream of the racoon dog stall are *in general* downstream of a big chunk of the western wing and the entire western wing respectively, so those mean little.])
Scott- I can roughly translate my factors into the grouping you use in your table.
Net priors using your grouping)
me: 12 you: 16
combo of all HSM/lineage factors
me: 1 you 0.002
FCS-ish
me 25 you 25
reasons WIV wouldn't do it
me: in priors you: 0.17
cover up success
you 0.5 me: did it?
other factor restriction enzyme pattern
me 70 you NA
The RE pattern is new, and I think you'll find my argument at least reasonable.
The big disagreement is about Worobey/Pekar. If I were to argue for ZW, I'd stay the hell away from Pekar, whose errors are now verging on something worse than "wrong".
Worobey does correctly point out that there was a spreading event at HSM, but so many specifics (lineage, internal DNA-RNA correlations, Wuhan share of relevant species (if any) don't fit that ZW has to rely on the non-market versions, which actually were generally considered before hand to have more of the priors than a market version- including by DEFUSERs.
Lots of people in the comments seem to have additional evidence one way or another that they want to add in. Maybe we need a coronavirus version of TalkOrigins.
I just cannot get myself to the point where I take anything being stated about this as reliable. For 2-3 years, everything that was being said about COVID by "experts" was either on what Zvi calls Simulacra level 2 (the public health establishment; saying X, despite ~X, to make you believe X and act in a way they want you to) or on what he calls Simulacra level 3 (Twitter users saying something not because THEY believe it's true, or even that they expect YOU to believe it, but just to signal that they're good non-racist Science-lovers). So even while I find Peter's points as you've summarized them to be pretty reasonable, he hurts himself IMHO by making the characterization of lab leak as a "conspiracy theory", because accusing an opponent of that also happens to be great tribal signaling. And the data on which all of this is premised comes from sources like the WHO which is clearly willing to lie for a variety of reasons.
So despite finding Peter's arguments here pretty reasonable, I can't get over the hump. I also, were I as dishonest as the PH establishment, would be inclined to just say it was a lab leak to induce public fear of gain-of-function research to help eliminate it. I can live with that having been wrong for this one virus, but still wanting to ban such research because next time it may NOT be wrong, but since most people don't reason that way then zoonosis being true is quite a blow to efforts to end this dangerous stuff. So let's give public health a dose of its own medicine and keep a nice healthy percentage on lab leak regardless.
Sneaking in late just to say this is my second ever favorite post ever, can I change my survey answer to NOT favoring lab leak? : )
Even after this excellent post, I find Michael Weissman's v5.7 analysis from March (https://michaelweissman.substack.com/p/an-inconvenient-probability-v57) to be the most compelling information available online to date, and it contains significant evidence updates from his v5.6 post that you linked in the write-up from January. I've kept an open mind through the years about Covid origins and found the Rootclaim debate to be especially interesting and cause for shifting my probabilities, but I have not encountered anything that has so significantly updated my probabilities than Michael's latest v5.7 analysis does. I'm curious if people have read that analysis in full and what everyone's thoughts are.
You seem to initially be interested specifically in Michael's Bayesian probabilities to compare his results to your table (which I think he posted recently in the comments above). I understand that you likely have dedicated dozens of hours to this post already and reading through a massive updated analysis is likely not especially appealing, but just looking at the probabilities alone does a disservice to the immense amount of research, evidence, and analysis that is included in Michael's write-up. For what it's worth, I've read an incredible amount of high-effort research and debate on this topic throughout years of following this, and Michael's research/analysis is the first one that I find convincing. In addition to more convincing Bayesian logic, his research includes crucial compelling evidence/information that came out after the debate (or was available but dismissed or potentially misrepresented at the time of the debate). I am trying to be as fair to both sides as possible and I honestly can't see how someone can read Michael's v5.7 post in full and come away with a 90% probability of Zoonosis. In my mental model, the two most likely reasons are that either people haven't read Michael's 5.7 analysis due to low exposure of a small blog, or I am fundamentally misunderstanding something.
Either way, I want to know what people think. If Michael's post isn't as convincing as it seems to me, I'd like to know why so I can update. If it is convincing, I'd like others to know so they can update. @Scott, I historically have placed a lot of value on your opinions and I'm genuinely interested in what your thoughts are here.
Thankyou for writing this up.
Personally, I don't have a lot of emotional/ideological investment in the issue so wouldn't sit through the debate but it great to get a detailed enough write up of the main arguments and their problems. I guess I should be over it, but I remain amazed by the level and vehemence of the responses here and elsewhere. Lab leaks remain possible whether or not it happened here so there's an ongoing risk anyway. There are plenty of risks. In the long term, I'm actually more worried about the intentional release of an engineered virus than an accidental leak.
The Rootclaim response in case it hasn't already been shared. Although the main issue to me is several papers have superseded the key papers relied on for the debate anyway. https://blog.rootclaim.com/covid-origins-debate-response-to-scott-alexander/
Peter had more at stake than Saar.
100k is about 0.05% of Saar's net worth. I assume it is a much much larger share of Peter's net worth. More to lose. More to gain.
The incentive for debate performance is massively skewed toward Peter in this case. Worth noting when analyzing the truth-seeking utility of this format.
Sorry if this is noted elsewhere, it is a big comment section.
This was such an interesting read. There are a lot of smart people here in the comments continuing the discussion, but I just wanted to chime in and say THANK YOU for writing something so compelling to read for somebody (me) who has no prior knowledge or real opinion on the subject(s). Legend ⭐