197 Comments

Interesting evidence indicating that R0 might actually be lower than Delta but with some immune evasion that allows it to spread in areas previously hit by Delta:

https://twitter.com/trvrb/status/1465364300936085506?s=20

Expand full comment

Came here to post a different tweet in the same thread.

Expand full comment
Comment deleted
Expand full comment

Paula came here to post the same tweet in all threads.

Expand full comment

I'm not really sure R0 is actually a meaningful thing to run a prediction market on. The problem is, when you dig into how R0 is calculated, it's not really a precise or well defined value. In theory its definition is simple enough but in reality, it has no biological basis - claims that R0 is this or that are always coming from models. And when you look at what the models do, well, R0 turns out to just be some kind of fudge factor or arbitrary coefficient used to make the model roughly match the data reported so far.

As a consequence I've seen models that yield totally different values of "R0" for the same virus on the same dates, simply because the model was given case curves from different cities. As such it cannot be genuinely said that this represents anything genuinely true about the virus itself. It's not like it's derived from RNA or careful lab experiments or anything like that.

Expand full comment

There are a lot of things in the world worth making predictions about which we can't nail down with the kind of scientific precision we would like. The solution is, rather than not trying at all, to just pre-specify as precisely as possible the method by which we will make the assessment in the end.

The fine print of the Metaculus prediction says:

"This will resolve on the basis of the mean R0 that is estimated for the Omicron variant according to the first credible systematic review that estimates this value. Such a systematic review should be peer-reviewed and should incorporate R0 estimates from at least 3 studies."

Metaculus users aren't being asked to predict "what is the R0 of Omicron", but rather, "what will the estimate of the first credible systematic review of R0 studies be?" And this is close enough to being the same question that it's still useful to us.

Expand full comment

Yes, the prediction market can be successful on its own terms if it's just trying to predict what academics think R0 is. That isn't quite what I meant by being a meaningful thing to run a prediction market on though. I'm just really skeptical the resulting value is actually useful in any way.

Expand full comment

It might be more meaningful to ask what the Rt of Omicron is, rather than the R0 of Omicron. R0 is the number of people each person infects *assuming no public health intervention.* But there are public health interventions!

Rt also has the advantage that you can estimate it with fairly minimal modelling assumptions. You need to know the interval between infections, or at least a distribution describing the serial interval. But aside from that, you can estimate it using only case counts.

Expand full comment

I don't see how that is more meaningful, though. Whether you're measuring number of people infected without interventions or with interventions, the value you get is going to be highly dependent on how much time the people in the area are spending breathing each other's exhales. Do a thought experiment: Town of 10,000 people, very small and congested, large families squashed together in small dwellings. Let's say there are no public health interventions. One person with the virus enters the town and we track transmission for a while, and using that info we calculate how many people the average infected person infects. Now imagine the same 10,000 people, still no public health interventions, but now the people are living all spread out over the prairie. Some people live 5 miles from the next building. No real community gathering places except a coupla general stores. No schools, even. Introduce one infected person. Virus is going to spread much slower, right? R0 is going to be way smaller. And if instead there are public health interventions -- like say 50 % of the population is vaccinated, 75% wear masks around people they don't live with -- the Rt is still going to be way higher in the congested town than in the prairie community. Neither R0 nor Rt is a measure that is independent of how much face time the members of the population you are studying have with each other. The value you get is going to vary GREATLY depending on, basically, how many cubic feet of their neighbor's exhales an average citizen of the community is breathing per day.

That is why it does not make sense to talk about the R0 of a virus variant, as though it is a property of the variant that is the same at all times in all places. A virus with a R0 (or Rt) of 6 in the congested town setting might have an R0 (or Rt) of 2 on the prairie. In a setting where everyone never get closer than a mile away from another person both R0 and Rt would be zero..

Expand full comment

That's the point - Rt is explicitly about a specific population at a specific time, while R0 claims to be about a population in a timeless sense.

Expand full comment

As you observe, Rt boils down to a fancy derivative of a case curve, which means such a market would have to specify the exact time period and time series (e.g. city, country) over which the prediction will be made.

Expand full comment

The problem here (and with almost all prediction markets) is you aren't predicting what R0 is, you are predicting what academic journals say R0 is. If you believe there is absolutely no bias in academic publishing then "what academic journals say" may be a reasonable proxy for "what is reality". If you believe there is a massive amount of bias in academic publishing, then "what academic journals say" may be totally unrelated to what is reality. For things like sports reporting, the reporting source is almost always a very strong proxy for actual results.

This knowledge of who is reporting tends to present itself in the prediction market quite well. A hypothetical market participant could believe that the *real* R0 of Omicron is negligible, but they may simultaneously believe that the academic journals will strongly prefer papers that have models that insight fear (more clicks/buys) and thus they may bet that R0 will be very high for this market.

Unfortunately, prediction markets are absolutely terrible at finding truth for this exact reason. They can do a very good job of predicting how some oracle will report in the future, but that is not necessarily correlated with what is true/real.

Expand full comment

So what you're saying is that 4 teams of epidemologists could collude to subtly fudge the numbers and make a killing on the prediction markets?

Expand full comment

Is there a prediction market with this prediction where you play with actual money?

Expand full comment

You realize you would essentially be betting on how many people were going to die/get sick.

I think a lot of people feel intuitively that such betting would be distasteful.

Expand full comment

Insider trading alone would be a massive problem if markets that bet on "what academics say" ever became worth significant real money. After all lots of people know what a meta-analysis is going to say before it gets published.

Expand full comment

Yeah, I agree, mostly.

R naught is a property of the virus in a given setting. It's dependent on the virus's intrinsic transmissibility and also on how much time people in that setting log hanging out breathing each other's exhales. It depends on patterns of living, working, and socializing, and also on precautions such as masking and ventilation. No matter how transmissible a virus is there would be no transmission in a setting where each person never came within a mile of anyone else. R nought would be zero.

So yeah, talking about R naught in general is like talking about the risk of somebody getting run over. It's highly dependent on how much time the person logs crossing streets and how many cars are in the streets he's crossing..

Still, if one specifies a locale, say Brooklyn, it would be meaningful to talk about the R naught of virus X in that locale. (It would vary some depending on weather, part of Brooklyn and some other factors, but probably not all that much.)

Expand full comment

R0 in theory is an intrinsic property of the virus, before adjustment for other factors like population density, weather, etc.

In practice epidemiologists have no way to actually measure this, so they end up making claims about R0 that are derived from pet models that are being curve fitted to the published PCR testing numbers. As such claims about R0 can easily conflict in major ways: it seems to bother nobody.

Can it be said R0 is a real thing at all, given the total inability to measure it or precisely define what it means? After all, as you astutely point out, it doesn't really make sense to talk about transmissibility in the absence of an actual context as if nobody was around and you were sealed in a vacuum jar, transmissibility of your viruses would be zero regardless of what the viral RNA was.

As such I am very dubious about all claims around transmissibility of viruses emanating from professors of epidemiology. It's just not a sound field. Values for R0 are usually on close inspection actually values for Rt that have been fiddled with a bit, where Rt is more or less some derivative of the case curve. Some models even take R0 as both an input and yield it as an output! Have fun figuring that one out ....

Expand full comment

I've always been told that R0 is the average number of people an individual infected person infects, before recovering, in a hypothetical population where no one has any acquired immunity. That doesn't sound like an intrinsic property of the virus, unless you have a background assumption that fixes the behavioral and environmental features of the human population you're talking about.

The fact that they use the names "R0" and "Rt" is suggestive that the classical model *does* assume fixed human behavior and environment, with acquired immunity being the only relevant parameter that varies with time to change the reproductive number.

Expand full comment

Yeah, and it's just dumb for the classical model to assume human behavior and environment are the same all over the world, right? Because they're not.

Expand full comment

I mean, it's not dumb for scientists to use models that involve known false idealizations. Sometimes we can know that the idealizations are false in ways that don't make much difference (as when we assume that the sun and the planets are the only gravitational bodies in the solar system and there is zero friction as they move, or when we treat the ocean as homogeneous and infinitely deep in modeling the generation of waves by wind storms) and sometimes even when we know the model is going to be bad at making predictions, it can still be useful for helping us to understand the types of impact different factors can have (for instance, we treat a pool table as frictionless to understand how hitting the cue at different angles will send the other balls in different directions, and we treat an economy as homogeneous and well-mixing to understand how changes in supply, demand, taxes, and interest will affect inflation).

Expand full comment

Yes, I understand about the need to simplify things to make useful models. But the simplification baked into the definition of R0 seems so radical that it interferes with the utility of the measure. R0 purports to be a measure of how many people an infected person will infect, but in fact number of infections passed on by an infected person depends on the environment and people's behavior in it, and there is a LOT of variability in those things across different settings. It's like treating pool tables as frictionless when some have smooth glossy surfaces and others are rough and gravelly and others are sticky with maple syrup.

Maybe it would be more useful to have a contagiousness measure where the setting is part of the definition: Number of people an infected person infects in a city of a certain population density. We could then accumulate a body of info about how variations in population density and other factors affect contagiousness in other settings. How dense is my setting compared to the setting on which the contagiousness measure is standardized? Use that as a multiplier . . .

Expand full comment

Where nobody has any acquired immunity, where prior/adapted immunity doesn't exist, where everyone recovers at the same rate and has an immune system of identical strength, etc.

Basically, how many spherical frictionless cows are infected by another spherical frictionless cow in unit time. It's an interesting thought experiment but not a real number that can actually be calculated and have any meaning.

Expand full comment

Have you no heart? I'm sure the number means quite a lot to those spherical frictionless cows!

Expand full comment

For those of us who are neither spherical nor frictionless, do we still need to wear masks?

Expand full comment

Can the glasgow coma scale be said to be a real thing at all?

Can IQs?

https://slatestarcodex.com/2014/08/11/does-the-glasgow-coma-scale-exist-do-comas/

Feels like this complaint leans in the same direction

Expand full comment

Well, IQ is a pretty abstract scale. IQ itself is a real thing on its own terms because it has high test-retest consistency, the methodologies for determining it are standardized, it seems to correlate well with other "intelligency" things etc. At least I believe those things are true - I'm no expert in IQ though. The problems here start when you begin claiming IQ is the same thing as generalized intelligence, which of course is a common but much more controversial claim.

R0 is on much less firm footing than IQ. There's no standardized methodology to determine it, nobody even seems to agree on the precise definition (see the multiple similar-but-different attempts in this thread), it's not even clear there can be such a definition, and different academics routinely calculate totally different values via totally different methodologies for what is supposedly the same thing about the same virus.

Expand full comment

Given that savant abilities tend to correspond with other "deficits", the general intelligence thing is tricky indeed.

IQ tests certainly dont measure certain types of intelligence- social intelligence, organizational intelligence, creative intelligence, others.

The things IG tests do measure might be correlated with those things, but thats a complicated question.

I have thought about someone from a western country who was taken into a nomadic hunter gathere society. Say they had some sort of intelligence test for the sorts of intelligence that was useful in their living conditions. Would high IQ scores correlate? I don't know.

I understand there is something sort of like global intelligence, but one has to define it carefully lest it covers modal nodes of intelligence it actually does not.

Expand full comment

Came here to say sth. similar. You need to define at least the location to make a meaningful statement about R0.

Once you do that, I think it makes sense to use this, as the R0 of different viruses (under similar conditions of population density and habit) is so different; so it allows you to compare in a given setting.

Expand full comment

Probably also a reflection of the health system's detection capability, to identify infected people, so the 'measured' R0 in a city in Europe (such as London) could have a different 'measured' R0 than a city with similar characteristics in the USA (such as New York).

Expand full comment

Wastewater monitoring can be used to control for that.

Expand full comment

The prediction market question should probably be "what R0 will (sensible authority) settle on" so you know which methodology you're guessing against.

Expand full comment

These two questions bother me. But writing good questions is hard. Prediction markets aren't supposed to address that problem, so I'm not sure I'm justified in being bothered. (Unless prediction markets are supposed to address that problem by asking more objective questions. Which they are, but that's not the only thing they're supposed to do.)

The second question on deadliness bothers me less. It seems like a real-world practical question that people want to extract from the literature, so it makes sense to take the literature as given. I don't like the particular formulation, but that's the basic problem of asking questions. Whereas, I feel like the question about R0 is more like doing science and it shouldn't be taking the literature as given, but should be pushing for predictions of more concrete things. Eg, the ratio of R of the strains in the existing population instead of the absolute R of omicron in a naive population (which is also the practically relevant question).

But, again, I'm conflicted.

Expand full comment

R0 might be impossible to calculate accurately, but the ratio between R0s of different variants is much more well defined and could be possible to calculate. So conditional on some value for R0 of the base variant, the R0 of another variant could be predicted.

Expand full comment

Calculate how? There is no consistent methodology.

Expand full comment

Agree there's no consistent methodology to measure R0. Here's an idea for measuring the relative R0 of 2 variants: How rapidly does one take over? Once variant Herbie starts showing up in a population where all the infected are hosting variant Batilda, how long does it take for Herbie to be the variant found in 80% of tests? Of course in most cases things will be more complicated -- there will be more than 2 variants circulating. But the math would still be doable. Maybe you could do it sort of the way chess rankings are done --everybody's ranking is a measure of how hard to beat they people they beat are, and how consistently they beat them.

Expand full comment

I think that sort of metric might make sense, although of course tremendously confounded by how much sequencing a country does. I believe the UK is responsible for half of all GISAID SARS-CoV-2 uploads for some reason.

However, it wouldn't be R0.

Expand full comment

Well, I'm not sure the differences among countries in sequencing would be a confound -- if by confound you mean a variable that distorts results. A country that doesn't do much sequencing would give us a skimpy data set, but not a distorted one -- right? Or is there something I'm not thinking of?

Expand full comment

Perhaps confounded was the wrong word. The problem is that most countries do very little or even no sequencing, and there are gazillions of variants. It's not true that there's Alpha, Delta, Omicron etc. That's more or less a press fiction at this point. Look at GISAID to see the true family tree. So you get a lot of .... variants, strains, mutations, whatever you want to call them ... that grow rapidly but then top out at a few percent and don't go further for some reason.

So they'd never reach 80% and thus this figure could only be calculated retroactively. But the only reason people care about R0 is because fraudulent pseudo-scientists like epidemiologists claim it's very important and a high R0 number means "exponential growth" and thus P.A.N.I.C. (mathematics needs to be put in a care home for abused children after all this). As a consequence you'd end up calculating a lot of very dubious growth values for tons of variants that look initially scary and then two weeks later their growth tapped out for some reason, and you'd be doing it on like 10 data points in the best cases.

Overall I find myself thinking R0 is a concept that should just be humanely put to sleep, like the rest of the junk field that spawned it. Even if it had some actual precise meaning and could actually be calculated, so what? What's the actual actionable outcome of knowing such a value? Well, there isn't one. It's not true that high R0 values = PANIC because you'd still be left in a situation where you can't model how long the virus will grow for, or how serious it is, and even basic questions like "what is the actual capacity of our hospitals" turns out to be mired in nonsense and mis-understandings (by epidemiologists!). So it can't tell you anything and it ends up with stuff like what we see today, where the doctor who discovered Omicron points out that the person who had it didn't even realize they had "COVID" at all (because they didn't, the symptoms aren't even close to a match) yet because of years of misinformation about exponents and R0, scientists and politicians immediately panicked.

R0 is in the end highly reflective of the profession that spawned it: fuzzy, worthless, used only to create unjustified panic.

Expand full comment

Agree that we are all walking around with a sense of the relative contagiousness of variants, but how do we arrive at that? My sense of variants' contagiousness is based on the accumulated headlines I've read. It's sort of an average -- well, a weighted average, because I trust some headline-writers more than others. And I'm not proud of that. Do you have a better calculation to propose?

Expand full comment

The test you proposed in the other comment is roughly what I had in mind, with adjustments for level of immune escape, and assuming an equal serial interval between variants, but these can be measured independently.

Expand full comment

Many seem to believe there's a case for omicron being *less* lethal than delta, based on near-zero-evidence throwaway quotes being shared widely on social media, best I can tell, out of wishful thinking. A metaculus question opened on this here:

https://www.metaculus.com/questions/8766/omicron-variant-less-deadly-than-delta/

Expand full comment

I wish I was smart enough to know wether or not this article means I should be more scared or not. (I'm assuming....yes? Be more scared?). I'm boosted. Does that help or we don't know? #YourDumbestSubscriber

Expand full comment

I'd say yes, be more scared of lockdowns, but no, be less scared of dying from it. And we don't know about the vaccines yet, but they won't hurt

Expand full comment

It's all at the level of "maybe could be bad" at this point, so far as I can tell.

Expand full comment

It's not the thought that counts, it's the data.

Expand full comment

You should be proportionally more scared. Existing vaccines likely work poorly against the Omicron variant, but "poorly" is not the same as "not at all". All that vaccines (and other preventative measures) can do is mitigate the risk of infection. So, if you were a hermit living in some remote cave in the desert, vaccines would be irrelevant to you, since your chance of infection is zero anyway. If you have a habit of licking strangers all day, then no vaccine would help you for long. But, assuming you're a normal person, you can alter your behaviour within reason to reduce your chances of infection. With the Omicron variant, it means that pushing your behaviour a little closer to "hermit" might be a good idea.

Expand full comment

You sound surer than many sources I'm reading that existing vaccines will work poorly against Omicron. Also, before you tell somebody how scared to be, you need to have some info about other factors: How contagious is the sucker? How virulent?

Expand full comment

Well, it has three known antibody-evading mutations (from experimental work on exposing mutated spike proteins to vaccinated/convalescent serum) but nowhere near as many antibody-evading mutations as that work needed to fully evade antibody response. (And of course even fully evading antibody response still leaves T-cell response which is much harder to test.)

So we should have high confidence (>90%) that Omicron erodes antibody effectiveness somewhat, and that it does not come close to eliminating it. (>99%)

I don't know what Bugmaster meant by 'poorly' exactly; it's one of those things that needs numerical precision to evaluate. I'd estimate top vaccine effectiveness will remain over 70%, but that's not really based on much data.

Expand full comment

Some of this seems quite wrong. Most obviously, the idea that a vaccine wouldn't help you if you went around licking strangers all day. If anything, the vaccine would help you most in that scenario!

More debatably, but still seemingly wrong to me, is the idea that vaccines "likely work poorly" against Omicron. No one really knows yet, but the evidence we have suggests that they may well work about as well as they ever did in protecting against serious illness. Which is the most important thing. So that seems like an at best overconfident, at worst very inaccurate characterisation.

Expand full comment

Data from Israel last night: Vaccines protect against Omicron about as well as they do against Delta. The unvaccinated are about twice as vulnerable to Omicron as to Delta.

https://www.mako.co.il/news-lifestyle/2021_q4/Article-0e660b77fe17d71027.htm

Expand full comment

> twice as vulnerable

Is that "twice as likely to get infected" or "twice as likely to get sick/die"?

My tentative understanding of Omicron, definitely subject to update, is that it is less deadly but spreads much faster.

Expand full comment

I wonder about that too. We need someone who can read Hebrew to go thru whole article and summarize.

Expand full comment

My gut feeling was that vaccines would protect at most 50% as well against Omicron as against the vanilla variant (and more likely around 25%); looks like I was in the ballpark ?

Expand full comment

Wait, no FDA approval?! That's outrageous!

Expand full comment

There is an interesting article arguing that the models being used for Covid are wrong, mostly because they ignore the structure of a population, the fact that A is much more likely to interact with B and C than with X, Y, and Z who are interacting with each other. He argues that the actual pattern of what has been happening cannot be explained with the current models. One of his conclusions is that the newer variants may not actually be much more infectious than the older.

https://cspicenter.org/blog/waronscience/have-we-been-thinking-about-the-pandemic-wrong-the-effect-of-population-structure-on-transmission/

Expand full comment

Why did you drop a letter from your name in your last few comments?

Expand full comment

It really is giving some of us fits. [no not really. but we are curious]

I mean if you were Grampa Simpson you could just say you can't use 'n' cause the Germans stole it.

Expand full comment

Not the real David F. would be my guess.

Expand full comment

That’s a good guess. Like the spam I get from Amazonn.con

Expand full comment

Except it exactly matches the real David Friedman's writing style, posting style, subject interest, economic positions and beliefs. It would be an amazing impersonater if true..... What would be the point? To save the real David from getting out of bed?

Expand full comment

And he's been here long enough that the real DF could've shown up.

It's a profile setting for substack, though.

Expand full comment

GPT-DDF

Expand full comment

I have no idea why the letter got dropped; I've been blaming Scott. It's probably somewhere on his floor or maybe in one of his pockets.

Expand full comment

Maybe he hid it somewhere in a logo like with that N in the slatestarcodex blog (because it was the gap for making it a perfect anagram for Scott Alexander)

Expand full comment

That does sound like him.

Expand full comment

This is really good. I've seen some people make claims that population structure would affect things, but they've done so at the length of a tweet or a Facebook comment thread, and not with any of this relevant detail. I would like to see a bit more about how sensitive this model is to certain structures of connectivity across the graph, and how things like actual transmissibility advantage of one variant over another and actual mild effectiveness of NPIs would appear in this model, but it makes a lot of sense.

I'm a bit surprised to hear him say that people have been denying the importance of this kind of structure. But this is partly because as far as I know, nothing in the US has ever been done on the basis of calculations of R or transmissibility advantages - perhaps in Europe people were trusting the simple models a lot to calculate those things.

Expand full comment

For what it's worth, I did some work about a decade ago on damage spreading in networks which is *somewhat* similar if you squint hard enough. The key finding is that the largest eigenvalue of an appropriately scaled adjacency matrix determines if you get percolation (i.e., a pandemic) or not.

Of course, the details are vastly different between what I worked on and disease spread, but I think some of the broad conclusions are probably applicable, which are that you can get significant spread when it's unexpected by naive models--that only look at average numbers of links--just by changing the connectivity patterns. In particular, assortative networks in which highly connected nodes are more likely to connect to other highly connected nodes will have damage spread more easily.

Expand full comment

Not quite true. There were people writing papers pointing this out in 2020.

https://nicholaslewis.org/why-herd-immunity-to-covid-19-is-reached-much-earlier-than-thought-update/

This was before the second wave so the argument about heterogenous connectivity is mixed in with claims about much lower herd immunity thresholds but the gist of it is there - the "cutting edge" academic models at the time assumed everyone was identical in every way. When people with actual standards and scientific ability started looking at them, they not only immediately noticed this problem (that somehow the field of epidemiology itself had spent decades not noticing), but also built alternative models that fixed it and realized it meant the original predictions were wildly off.

Expand full comment

This doesn't seem like an even vaguely accurate summary of the state of epidemiology to me. Google "network epidemiology" to see papers, journals and textbooks going back at least two decades that attempt to build models accounting for network structure.

The Imperial model which was the focus of UK policy at the beginning of the pandemic was an agent-based simulation based on multiple factors (age, location, transport links, etc), and simulating contacts on a non-homogeneous network. Obviously some relevant information was left out (that's the nature of modelling), but the idea that epidemiologists had somehow just not noticed that homogeneous SIR models don't perfectly predict reality seems very clearly false.

Expand full comment

I don't think that model had non-homogenous contact networks, which is what makes the big difference here, but I might be mis-remembering. It did do some basic age stratification and had different "R0" (or some sort of R) values for different "places".

I've never seen any admission in epidemiology papers that their models fail to predict reality. Although they produce new models all the time, it's not like some fields where there are open competitions, robust accuracy metrics etc. Actually I rarely if ever see any comparisons between modelled predictions and what really happens, which is why as a group they've developed a reputation for being constantly wrong and yet acting as if they weren't.

Expand full comment

This is just really uncharitable. Let's take as an example the paper that describes the model I mentioned above. The full text is here: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7095311/

Here's a quote from the final paragraph:

"Although more detailed model validation and parameter estimation using data from past pandemics should be a priority for future research, it will be impossible to predict the exact characteristics of any future pandemic virus."

As for the details - the model itself is agent-based. It predicts individual contacts of the simulated agents, assuming the contacts take place in different types of location (workplaces, schools, households, and 'other' if I'm reading it correctly). This necessarily results in a non-homogenous contact network (kids associate in schools, adults don't).

The idea that epidemiologists weren't aware of the limitations of models assuming homogeneous contacts is, as you imply in your first post, ludicrous and as such you should be sceptical of it. My general impression is that they have become significantly more aware of it over the past decade, and models have started to try to take it into account more and more (basically, as the computation power and data to make such models useful has become available)

Here's a paper from 2019 which deals with the question of non-homogeneous mixing in equine populations:

https://www.nature.com/articles/s41598-019-40151-2

Here's some work considering the effect if the structure of the network changes over time:

https://www.sciencedirect.com/science/article/pii/S1755436518300173

To quote the introduction of that second paper, from 2018: 'a full consideration of the often profound effect of heterogeneous contact structure on infectious disease dynamics has really only become widespread more recently, and especially since the adoption of methods often originally conceived in the context of social network analysis'

Sometimes epidemiologists choose to use the simple models anyway, because they're faster to compute, and accurate enough, but they (or at least some of them) very definitely are aware of the limitations.

I don't doubt that some epidemiologists overstate the case for some simple models, and I don't doubt that sometimes the models are used badly, but the generalisations that you're making seem extremely uncharitable and honestly, I don't think they're very accurate.

Expand full comment

I don't mean a generic disclaimer of the form "of course, models aren't perfect". Every paper has those but who cares? What I mean is the actual scientific process:

1. Hypothesis.

2. Experiment or observations.

3. Refinement, abandonment or acceptance as theory.

Epidemiology seems to consist near-exclusively of (1). Where are these papers by Imperial College that dissect and analyze their previous failures, which have been numerous and enormous over a period of decades? There are none. Where is the admission that the model they put in front of governments for COVID was totally unvalidated, and unfit for purpose? There has never been such an admission.

Perhaps you think it's unfair to harshly judge the whole field on the basis of Imperial College. In fact I've seen lots of papers that are like their work, but, OK, other than John Ioannidis (who has been treated with incredible disdain for doing so), where are the epidemiologists condemning Ferguson's team and furiously demonstrating how they aren't all like that? Where are the demands that "scientists" who promote ridiculous, totally unvalidated models that produce incorrect predictions be fired or otherwise penalized? There's nothing because they're all at it and don't see anything wrong with it.

As previously mentioned, Ferguson's model is very concerned with R and R0. It took such a value as input. Most of this discussion is about the fact that no such value can be determined and even what it means is totally up in the air. These guys have been working on their models, publicly funded, for more than 20 years - the world can and should expect better than this especially given the horrifying impact these incorrect model predictions had (there was never going to be a single massive wave and they knew it because epidemics have never worked like that).

Expand full comment

Yes, the Imperial model was complicated, but it was worse than a raw SIR model. Pretending that epidemiological study of networks doesn't exist is overly charitable.

Expand full comment

I think that's a different argument, and one I was making in a less detailed form. If some people are more likely to catch the virus than others, for either biological or behavioral reasons, then the more vulnerable are being selectively infected, so the average vulnerability of the population is falling over time, so her immunity is earlier.

http://daviddfriedman.blogspot.com/2020/08/covid-invisible-elephant-in-room.html

Expand full comment

I'm glad you found the post interesting!

I agree that it would be interesting to study more systematically how the properties of the network affect the dynamic of the epidemic. This is a natural development of the work I did in this post. However, that's a lot of work and, in that post, I just wanted to present the basic idea and some results showing that what you would expect intuitively can in fact happen in simulations, as well as explain why I think it matters.

On your second point, this is something I already thought before and some of the reactions to this post have only strengthened this view, but I do suspect that the difference between how things went down in the US vs. in Europe explains in large part this difference of perception. I have been in Paris since the beginning of the pandemic and, in France, the kind of simplistic models I criticize have played a very prominent role in the justification of the government's policy, if not in the elaboration of that policy itself although it clearly had some impact on that as well.

For instance, the leader of the group who published this ridiculous paper (https://www.medrxiv.org/content/10.1101/2021.02.14.21251708v2) was invited to a press conference organized by the French government in February during which she pretended that it showed that curfews were very effective, which the government explicitly used to justify the continuation of that policy, even though the methods in that paper can in no way support that claim. (For more on why, see https://necpluribusimpar.net/lockdowns-science-and-voodoo-magic/ and https://cspicenter.org/blog/waronscience/the-british-variant-of-sars-cov-2-and-the-poverty-of-epidemiology/ in addition to what I explain in my post on population structure.) Throughout the pandemic, every time Macron announced new restrictions, he explicitly cited projections by this group or another at the Institut Pasteur.

I think modeling also played a major role to justify policy in the UK. From what I can tell, it was very different in the US, where apart from the cubic model fiasco — which didn't really matter anyway because the power to order restrictions to contain the pandemic belong to the states — I don't think modeling played a major role in political discourse. But in France and many other European countries it was very different.

Expand full comment

I imagine this must be a lot of work! Thanks for posting some of the things on Github. I unfortunately don't have much experience with this kind of thing, but this post was so interesting that I'm wondering if it will be able to get me to try to teach myself how to use some of it over the winter break to see if I can get a bit of understanding.

Some questions that suggested themselves to me while reading (just in case one or more of these turn out to be easy to address, or if anyone other than you is reading who can work on these models), in addition to the ones about what happens if there is some slight effect of NPIs, or slightly different transmissibility of the variants:

How sensitive is this to the seeding mechanism? If seeding happens uniformly in all sub-populations and sub-networks, then you automatically get a transmission advantage for new variants. (New variants will be equally spread across all sub-populations, while old variants will be disproportionately concentrated in sub-populations that have a high current infection rate, and thus are lower transmission for all variants.) But seeding by new variants that have become common overseas shouldn't occur uniformly, but instead should have some patterns of their own. If we treat your model as a model of the *world*, with no seeding, rather than a model of a country with external seeding, then this effect might go away, since new variants will be more likely to appear in sub-populations that have more cases. However, if we ignore new variants until they account for 100 cases, this will select for paying attention to variants that, by chance, are present in sub-populations with high current transmission rates. Does this restore the effect?

In your model, the seeding switches from one variant to another, and the second variant ends up taking over much faster than one might naively expect. If both variants continue seeding equally, how often does it occur that the new variant ends up becoming more common than the old variant? Or in a model with no seeding, but just occasional random mutations into a new variant (without any transmissibility advantage) how often does the new variant end up taking over?

How small of a transmissibility advantage yields transmission advantages that seem qualitatively similar to the data we have for alpha and delta?

I really like the idea you have of explaining geographic correlations by having networks of sub-populations that don't line up with geographic networks. It fits with a lot of things I've been thinking for years (including many topics Scott writes about) - city center networks in different cities are often going to be more closely connected to each other than to the suburban networks immediately surrounding them, and blue tribe/red tribe/grey tribe networks will be connected to each other in different ways than they are connected across these lines within a geographic area. And anecdotally, I noticed that I had several friends (around the world) who were infected with covid in March/April 2020, but then none (as far as I know) until April 2021, when a friend in Michigan, a friend in Toronto, and a friend in Paris all got infected within a few days of each other. It would make a lot of sense if these people are all multiply connected through social networks that are geographically dispersed.

But it does seem to me that even if this sort of tree-like hierarchy of network and sub-population interconnections has some geographic dispersal, that there should also be a cross-cutting set of geographically local connections. It might not be too hard to add extra connections across sub-populations that are geographically local, but maybe it is. It would be interesting to see how sensitive some of the big results are to the tree-like structure where sub-populations are grouped within a network, if instead we have sub-populations in a bit more complex of a network (like a small-world graph, for instance).

This also raises a deeper question. The idea of a population as a well-mixed and homogeneous network is obviously very unrealistic. But conversely, the idea of sub-populations as being mostly self-connected with only occasional connections within a larger network, and only very rare connections outside that network, also seems quite unrealistic. The latter model might well be a very good model of what happens during a period of very tight "lockdown" (whatever that means to people). I think you did one investigation that shows that with enough connections across these sub-populations, you end up with something that looks very much like the homogeneous model, but it would be interesting (and presumably extremely difficult) to see what happens between. And it also makes me wonder what happens if we model "lockdown" with a switch between the two types of dynamic.

Expand full comment

Yes, modelling was the single key factor that converted the UK from a relatively Sweden-style relaxed approach to harsh China-style approach. Once Ferguson made his graph of a single giant curve totally swamping all hospitals, it was over because nobody in government seemed to be aware of his team's numerous prior failures, and suddenly all sorts of academics popped up claiming ICL were the "best epidemiologists in the world".

Lots of people in the general public caught on within months that the man was a ribald liar - despite claiming some sort of apocalypse was inevitable even if everyone did lock down, he was immediately caught breaking lockdown rules to go sleep with his lover, who not surprisingly was a far left extremist (extinction rebellion). Clearly he didn't actually believe his own predictions and the nation found out why later on, much to its own cost.

Expand full comment

My understanding of the rationale behind lockdowns was to reduce the homogeneity of social structure, to give us contagion dynamics that look more like the waves and less like the big bell curve. I don't want to take responsibility for anyone else's messaging on that point, but that's how I understood from the start: reduce the number of edges on the graph, and reduce their strength, as much as possible, to reduce the degree to which infection was GEOGRAPHICALLY correlated (since that's also how medical resources are distributed).

Journalists may have made a hash of the point but who cares?

Expand full comment

We saw really quite decent evidence for at least some of this population-structure based spreading in the last UK wave: it produced a very significant peak in the teenage population, and a corresponding smaller peak in the age group corresponding to their parents. The younger and older adults saw much smaller growth.

See the "Cases by specimen date age demographics" heatmap here (scroll down a bit): https://coronavirus.data.gov.uk/details/cases?areaType=nation&areaName=England

Expand full comment

Spin-off question: Would it be theoretically possible for a variant to be, for all practical purposes, a covid vaccination? So it would be a virus that kills about 1 person in a million, but mostly causes no symptoms or very mild ones -- and confers immunity to covid for the next few months to all who have been infected. And if that's possible, mightn't it be possible to tweak the variant so as to make it so hugely transmissible that everybody gets it, even the animals who can be infected by covid? If that happened, could we truly and permanently get rid of all variants except the benign one?

I'm sure it's very unlikely that such a virus will pop up, but still would like to hear from knowledgeable people whether such a sequence of events could happen if such a virus did appear.

Expand full comment

My guess is that it is way beyond existing biotechnology to create a variant that (1) is mostly asymptomatic or mildly symptomatic and (2) confers immunity to existing variants of SARS-CoV-2 and (3) is at least as transmissible as Delta. Being transmissible requires hijacking cellular machinery to replicate itself, and that's the kind of thing that triggers the immune system and usually provokes symptoms.

Expand full comment

But there already are viruses that work this way. We call them "the common cold", though it's really many separate viruses.

It would probably require a fair amount of luck to have such a Covid-19 compatible virus appear, but it seems possible to me.

Disclaimer: I have no virology or related expertise at all.

Expand full comment

Not a virologist, but it seems that you'd need to restrict infectivity to the upper respiratory tract (as opposed to systemic infectivity that COVID has) while keeping the spike protein structure largely intact, and the mechanism that you'd use for that need to be robust enough so that a few random mutations here and there would not cause it to become systemic again. Even if it were possible, it seems very easy to make matters worse while developing such a mechanism and having an accidental lab leak.

Expand full comment

If there's "gain of function" research, why not "loss of function" research?

Expand full comment

At that point why not just have vaccines? I guess the answer is (i) vaccines you have to go out and physically jab people, and (ii) some people don't like vaccines. On (i), fair enough but at this point the infrastructure required to physically jab people isn't the main bottleneck, and on (ii) I don't think the people who dislike the vaccine will be happier with "it's like a vaccine but it spreads through the air so now you CAN'T avoid it!"

I'd also be worried that it mutates into something deadlier, apparently that can happen a lot!

Expand full comment

I think you'd still have to go physically jab people - as contagious as COVID is, it's nowhere near contagious enough to give the high % coverage you want by natural spread.

Expand full comment

Maybe we need a more benign version of the Rage Virus in *28 Weeks Later" . . .

Expand full comment

The Mild Annoyance Virus?

Expand full comment

Yeah! Or the Peevishness Pox

Expand full comment

It's sort of amazing that even at the low points of the vaccination program, there have been over 700,000 people getting vaccines per day in the United States, while the highest recorded number of infections was about 300,000 per day in the middle of last winter's wave. There are surely more actual infections than the recorded number, but probably since March or April, there has likely not been a single week where more people were infected than got vaccinated.

Expand full comment

I think that at least some consequences of the virus (sneezing being a prime example) serve to enhance infectivity. So you wouldn't be able to both perfectly maximize infection and minimize damage. My guess is that the more harmful parts of COVID would somehow also enhance infectivity (since for example asymptomatic cases seem to have lower infectivity rates from what I last read, but that may be out of date), to the point that it would be difficult to make a low-damage competing strain.

Expand full comment

This is sort of how the first vaccinations worked: using cowpox as a vaccine for smallpox. So like, it's *possible* for such a thing to happen by chance.

But we had to manually infect people with cowpox, it didn't outcompete smallpox on its own. And cowpox only had to be safer than smallpox, which is a much lower bar to clear.

Expand full comment

Yes, it'll hopefully evolve into just another nasty cold within a number of years. no, deliberately exposing the population to infectious diseases is a bad idea, even if you think they're pretty innocuous.

Expand full comment

That's the _Medical Police_ season finale!

Expand full comment

That's pretty much what cowpox is for smallpox, and is why it's called a vaccination.

Expand full comment

A couple of people have mentioned the original small pox vaccination. But a better match for this is the oral polio vaccine which is an attenuated virus that is capable of (at least in limited circumstances) of spreading from person to person. This helped increase vaccination coverage in difficult to reach areas. But it can also mutate to more severe forms (see vaccine derived polio, though note is still typically milder than wild type polio) and at one point such forms were the dominate circulating strains in certain areas. As polio has gotten closer to eradication this vaccine gets phased out in favor of the injected killed virus form.

Expand full comment

+1 for not *completely* disregarding the "hugely transmissible" part of Eremolalos's comment, unlike nearly everybody else.

Expand full comment

+2 for noticing.

Expand full comment

That's the principle behind the cowpox->smallpox vaccine approach, so it seems at least feasible?

Expand full comment

Theoretically, yes, practically, no.

Virulence is generally a side-effect of infectiousness. Cow pox and traditional attenuated/live vaccines are mild viruses. They injection causes enough of an infection to produce an immune response in the individual, but it doesn't reproduce enough to harm the individual or spread to other people. Often live vaccines evolve to spread faster, but when they do, they are harmful like the original form.

I have read that live polio vaccines rely on the target infecting other people who were not directly vaccinated. But I don't think it spread very far; rather it has R<1, which means it spread, but burns out.

It is plausible that omicron is less infectious and less virulent than delta, but with more immune escape, so that in a naive population it would be outcompeted, but in existing highly exposed populations it is spreading. This would be mildly bad for the people who were already exposed and get a second mild illness, but potentially good for the people who have so far gotten nothing to get omicron rather than delta.

Expand full comment

There's evidence that something like this has happened naturally in the past. Despite its name, the 1889-1891 "Russian flu" pandemic may have been caused by a coronavirus, not an influenza virus. One candidate for its identity is a coronavirus that today is implicated in up to 30% of common colds. See: https://sfamjournals.onlinelibrary.wiley.com/doi/full/10.1111/1751-7915.13889 for more details.

Expand full comment

Do we *know* the virus itself has become virulent? Or might it be that almost everybody catches it umpteen times while young but if someone somehow managed to catch it for the first time when they're 70 it'd be as bad for then as it would have been in 1890?

Expand full comment

Sure, it could definitely happen. It's definitely occurred to me that the 'common cold' is actually the equivalent of [Carcinisation](https://en.wikipedia.org/wiki/Carcinisation) in viral evolution. There are over a hundred endemic viral infections with symptoms and contagiousness so similar that we lump them all into the same disease; it's clearly a pretty stable evolutionary equilibrium. It's entirely possible that COVID will also eventually create a 'common cold' version of itself, with all the advantages that brings.

However, it's worth noting that we'd still probably be quite a bit more worried about it than a normal cold. The mutational distance from the hypothetical 'COVID cold' variant to something legitimately deadly with systemic infection would be much smaller than for typical colds. We'd probably keep a high alert on monitoring it, and push vaccination for it into the mandatory vaccine schedule to prevent its inevitable return as a deadly disease.

(Of course, actually tweaking such a disease and deliberately increasing its transmissibility would be folly of the highest order, even if we could do it. You don't WANT to make a disease that's a short mutational distance from deadly, highly transmissible. Its eventual deadly offspring would likely retain that transmissibility and be horrifying.)

Expand full comment

> Metaculus didn’t want to wade in to precise lethality statistics, so they just asked for a yes-or-no answer on whether it would be deadlier than Delta. Forecasters say there’s a 34% chance it will be.

Is that "deadlier on a per case basis", or "deadlier en masse"?

That is, if I catch Delta will I breathing a sigh of relief that at least I didn't get schwacked with something really horrible like Omicron, or is it that way more people will catch Omicron than Delta so Omicron will rack up a higher body count over all by increasing the sheer number of infections?

Because... I'm given to understand that that's how endemic diseases work. It starts off lethal, then millions of millions of generations later the virus finds a sweet spot of being super infectious but relatively chill about killing the host. If Omicron spreads like Gangnam Style but is, say, half as likely to kill you... isn't that a good thing, overall?

Somebody, anybody, feel free to correct me harshly if I'm way off base.

Expand full comment

>Is that "deadlier on a per case basis", or "deadlier en masse"?

I'm pretty sure it means "deadlier on a per case basis", i.e., "probability of death, conditional on getting infected". But looking at the wording of the question, it's not very explicit.

Expand full comment

> then millions of millions of generations later the virus finds a sweet spot of being super infectious but relatively chill about killing the host.

The virus doesn't really *care* if the host dies or not, except in the fact that it gets to spread.

For super serious diseases like Ebola, they would benefit from being less deadly, because when you get sick you know it and hide away from others.

There is lots of room for covid to be either more mild or more severe while still becoming more infectious.

Expand full comment

Sure, I mostly agree, but I think there is still a selection force toward a less severe virus (subject to the actual constraints of chemistry and biology of course.) There are certainly symptoms that COVID would benefit from losing. Lost of smell/taste is an obvious one; if people have a harder time distinguishing COVID from the flu, it could spread more easily. Similarly, the general fatigue from COVID reduces the sociability of its host; a bad trait, evolutionarily. Same with a fever; easy identification of illness without an associated upside.

There's a reason that there are over a hundred cold viruses. 90% of the things that increase infectiveness are respiratory. Almost all the symptoms from a systemic infection like COVID don't contribute to contagiousness of the host. A variant of COVID that only infected the nose and throat (rather than the whole body) would probably be much more contagious on a host-behavioral level.

Expand full comment

Is the deadliness question about whether it kills more people conditional on their being infected, or just that it kills more people altogether?

Expand full comment

My reading of the more detailed explanation is "conditional on it being infected": https://www.metaculus.com/questions/8757/omicron-variant-deadlier-than-delta/

Expand full comment

Do the prediction markets say anything about how likely it is that Omicron will evade current vaccine protection?

Expand full comment

Can I ask when you pulled those graphs? Because the numbers on Metaculus look substantially different now. The median prediction on mortality is now 25%.

Expand full comment

Is it reasonable to think:

1) The original covid strain is deadlier than most pathogens (and most coronaviruses)

2) Omicron has more mutations than other variants and is therefore the least like the original strain

3) We should expect reversion to the (less deadly) mean for Omicron which may be more like an average pathogen and less like the original covid variant

4) Therefore, all else being equal, we might expect Omicron to be less deadly?

Of course, all else is probably not equal. Notably, there's reason to believe it would have better vaccine escape, and if it's better at spreading that may be a sign that it has qualities that would make it deadlier. (Better at evading immune response? Better at replicating quickly?) But perhaps there's some reason to think these scary features may be offset by less deadliness otherwise? Maybe that's just wishful thinking, though.

Expand full comment

I think you are wrong at least about 1). Look at SARS on wikipedia, the 2002-2004 outbreak of CoV-1 had some 8000 cases, with >700 deaths. I think this ratio is way over twice the lethality of COVID-19, and we were lucky that it did not spread as widely. (Note that I may be wrong about their relation, and also perhaps that initial rates of death might be higher due to the novelty, or perhaps we are doing better now due to advances in medicine. However I think we can agree that we did not get an uniquely lethal variant in ‘19. (MERS seems to have a cfr around 34.4?!))

Expand full comment

Covid-19 is almost certainly the third or fourth most deadly coronavirus to humans that has affected people in the last century or so. It's far more deadly and dangerous than the majority of them. (We've known about four of them that cause "common colds" for a while, but two of them were only detected in the 1990s, and there just hasn't been a huge amount of virus sampling to determine whether or not there are many others.)

Expand full comment

For the intrinsic deadliness that bortrand asks about, covid-19 is much less deadly than the common cold coronaviruses. The difference is that we get them as children and build immunity, whereas people are getting their first exposure to covid-19 as adults.

Expand full comment

That actually does sound plausible, though I don't know how much evidence we have for that. The best evidence for this is the hypothesis that OC43 might have been the cause of the 1889 pandemic, but this is definitely far from conclusive.

Expand full comment
founding

I'm skeptical of the claim that we *all* get OC43 as children; there are too many varieties of the common cold for everyone to get them all, and even if OC43 is one of the more common strains, I'd expect a significant number of people to avoid it by luck (or maybe geography). In which case, there would be a significant number of people first contracting OC43 as adults, and enough of them becoming really sick or dead that this would be noticed.

But that needs more careful analysis than I expect anyone is likely to give it any time soon.

Expand full comment

There are maybe 200 cold viruses, but the 4 coronaviruses are the common ones. 10-40% of colds are said to be from coronaviruses. In a sample of 105 people, no one tested negative for antibodies for OC43, 1 person for 229E, 2 for NL63, and 8 for HKU1:

https://pubmed.ncbi.nlm.nih.gov/20943876/

Expand full comment

Metaculus says it will resolve the claim based on "the first credible systematic review that estimates this value," citing Liu and Rocklöv (2021) as an example of the sort of thing they consider a “credible review.” But the methodology of Liu and Rocklöv seems atrocious: they write that across five studies they have identified, "the basic reproductive number for Delta ranged from 3.2 to 8, with a mean of 5.08." This has the following problems:

(1) The five studies are one article in English about Guangdong, one article in Chinese about Guangdong, two white papers about the UK, and one MedRxiv preprint about “China”. The five studies all used different methodologies for estimating R0. Liu and Rocklov say nothing about the strengths and weaknesses of these methodologies. Only one of the five studies indicated an explicit confidence interval (95% CI: 2.0–4.8).

(2) The reported “mean of 5.08” is simply the arithmetic mean of the midpoints of the R0 estimates of the five studies. Given the disparity of methodologies, calculating such a mean in the first place seems meaningless, and reporting it to three significant figures is ludicrous. The fact that this went through to publication does not speak well of the peer review process at the “Journal of Travel Medicine”.

(3) It doesn’t even make sense in the first place to talk about “the” R0 for a virus or variant — R0 is a parameter used in certain models for describing the dynamics of a pathogen *in a particular epidemiological situation*. Liu and Roclov half admit this, writing: “Given that the reproductive number in the studies identified here was estimated at a time when most countries still enforce a variable extent of lockdown measures, there is a risk that the real reproductive number may be even higher than the estimated 5.08.” But they fail to realize that the same logic implies that there is no such thing as the “real reproductive number” of a virus: there is only a range of estimated R0 parameters used in various epidemiological models to fit the observed case data in particular times and places.

(Epistemic status: I am not an epidemiologist, and I will be happy if anyone who knows what they are talking about can tell me why I’m wrong.)

Expand full comment

You're not wrong and there's a discussion up-thread about this exact problem.

Expand full comment

> It doesn’t even make sense in the first place to talk about “the” R0 for a virus or variant — R0 is a parameter used in certain models for describing the dynamics of a pathogen *in a particular epidemiological situation*. Liu and Roclov half admit this, writing: “Given that the reproductive number in the studies identified here was estimated at a time when most countries still enforce a variable extent of lockdown measures, there is a risk that the real reproductive number may be even higher than the estimated 5.08.” But they fail to realize that the same logic implies that there is no such thing as the “real reproductive number” of a virus: there is only a range of estimated R0 parameters used in various epidemiological models to fit the observed case data in particular times and places.

I want to address only this point out of the good points you made. There is (obviously, IMO) some 'contagiousness' trait of a given disease. For instance, we know that measles spreads extremely rapidly through unvaccinated communities with little existing immunity, while a new flu (say, H1N1 when that came out a few years ago) can take weeks or months to cover the same community. So (to use Eliezer's map/territory metaphor) there's a real trait in the territory that is a property of a specific variant of disease which affects how quickly it would spread in a given community. The fact that the speed of spread will vary between *communities* for all viruses (due to population densities, precautions, behavioral differences, etc.) doesn't eradicate the fact that the speed of spread will vary between *viruses* for a given community. They both contribute.

R0 when used as a property of the virus is a crude estimate in our map of this real trait in the actual territory. It is definitely a meaningful thing to try to measure in spite of the fact that it's horribly confounded.

Expand full comment

Right, I have no quibble with the idea that some variants of COVID are more contagious than others, and that the differences in contagiousness can be reasonably expressed in (ranges of) R0 values. My problem is with the framing by the Metaculus people: the idea that it is a sensible thing to bet on whether the R0 of omicron is 4.2 or 5.1 or 8.3. They get around this by saying they will resolve on the "first credible systematic review", but they seem to be unable to identify whether a published review is in fact credible or systematic. (Of course, Metaculus being unable to identify a credible review is somewhat less of a problem than the "Journal of Travel Medicine" being unable to identify a credible review...)

Expand full comment

I am new to prediction markets as a tool. I understand that Metaculus is probably not Wikipedia, i.e. it's not like a few uninformed but enthusiastic (or informed and malevolent) actors could significantly alter it, but who are the predictors that actually make predictions there?

Expand full comment

Readers Like You. (mostly a bunch of randos who just want to predict a lot and care about what a brier score is)

Expand full comment

Kalshi has a yes/no market on whether Omicron will make up over 1% of US COVID infections by the new year:

https://kalshi.com/markets/VARIANT-002

Currently trading at 71% ”Yes”.

Expand full comment

I'd be surprised if it wasn't. If (a) Omicron has any advantage at all over Delta in its ability to spread in a moderately well vaccinated population like the US, and (b) there are already seed cases within the US, then it is hard to see how the US wouldn't be at the necessary ~700 cases/day by the end of the month.

Given the number of cases already found in Europe, and the fact that the US had direct flights to Johannesburg, it would be shocking if (b) wasn't true. So the 30% is the chance that Omicron turns out to be a fizzer that can't actually outcompete Delta even in a well-vaccinated population.

If that's the case then I don't understand why Omicron has been spreading in South Africa while Delta has been receding. But on the other hand, I don't understand why Delta has been receding in South Africa since July anyway despite low vax rates. Actual viral dynamics are weird.

Expand full comment

The US federal government has purchased some 2 billion CoViD-19 vaccine doses. Today, the CDC changed their guidance from adults _may_ get a booster to adults _should_ get a booster. This is good, but instead of distributing booster doses to every American who wants one, would it be a more rationally selfish course of action for the US government to distribute those doses to people in the half-dozen countries that the Omicron variant originated from, using the logicstics + medical muscle of the US military?

It sounds like places with large HIV+ (i.e. sort-of-healthy-but-immunocompromised) populations are great places to breed new variants, and it seems infeasible to prevent highly-transmissible new variants from spreading to the rest of the world. So, are our vaccination efforts best spent on places most hospitable to virus breeding?

Expand full comment
Comment deleted
Expand full comment

It turns out that there are crazies everywhere, and that a bunch of them have access to twitter/facebook/whatever and can absorb crazy from more developed countries.

From the news, it looks like the SA government is specifically thinking about having vaccine mandates (which will no doubt be as popular there as in the US).

Expand full comment

Don't blame developed countries for African public health craziness. We weren't the ones who came up with "Have sex with a virgin to cure your AIDS", they came up with that all on their own.

Expand full comment

What has that got to do with covid? The craziness around covid in South Africa (such as can presently be discerned) seems to mirror, and take arguments directly from, the anti-vax movement in America.

Expand full comment

It's not only South Africa, it's the whole continent, and if the reasons given reflect the western antivax credo, I believe it's more like convergent evolution that a proof of influence...Because one thing is clear: in Africa, if you say "take a jab because the whites say this injection is good for you", you will get a large number of unconvinced people. Basically, you have 4 ways to get them convinced: pay them, force them, trick them, or say it's for something they scare more than the white playing a dirty trick - and the covid is far from scary enough, especially when you have HIV and Malaria around. There is maybe one other way: the injection coming from a source they trust....But not clear which it would be : the local government is usually no more trusted than the whites...BTW a classic trick would use reverse psychology and say the vaccine is only for the rich/whites, but also create some fake black market of vaccines....A dangerous path to try....

Expand full comment

It's not only Texas, it's the whole country, and if the reasons given reflect the African antivax credo, I believe it's more like convergent evolution that a proof of influence...Because one thing is clear: in America, if you say "take a jab because the Democrats say this injection is good for you", you will get a large number of unconvinced people. Basically, you have 4 ways to get them convinced: pay them, force them, trick them, or say it's for something they scare more than the government playing a dirty trick - and the covid is far from scary enough, especially when you have the opioid crisis and cancel culture around. There is maybe one other way: the injection coming from a source they trust....But not clear which it would be : the local government is usually no more trusted than the fed...BTW a classic trick would use reverse psychology and say the vaccine is only for the rich/liberals, but also create some fake black market of vaccines....A dangerous path to try....

Expand full comment

Any thoughts on what the federal government intends to do with 2 billion doses of COVID-19 vaccine?

Expand full comment

Put them in cluster munitions and vaccinate large crowds in a single pass.

Expand full comment

Early on, before any vaccines were approved, they signed contracts with four or five major manufacturers for enough doses to vaccinate every single American even if only one of them turned out to work. I think the plan has always been to donate the excess to other countries once it's clear that they're not needed, but it's unclear when that might be.

Expand full comment

It wasn't only that the doctor in South Africa noticed young people have less severe disease which would be expected. They also didn't lose their sense of smell and taste which normally happen even with mild covid. It could indicate less severe variant. Of course, it might be just wishful thinking. And yet I don't like so many headlines that it is time to freak out. Especially when we have zero indications that it is more severe either.

While Noah Smith correctly says that restrictions like border closures and other nonpharmaceutical interventions are now useless, politicians and most other people have very little attention span and will only hear the first line “now it is time to panic” and will introduce all of those restrictions again. We should have written the headlines and the first paragraphs differently – something among the lines that omicron is no worse that what we have and banning flights is even less useful, and only then going into details how it works.

Expand full comment

> While Noah Smith correctly says that restrictions like border closures and other nonpharmaceutical interventions are now useless

No, they're not useless. As Zvi said, they could be exactly what we need to buy ourselves time.

Noah said we can't *stop* Omicron, because it's already Out Of (South) Africa, and that's right, but we could push out the date when we (say) pass 1 million Omicron cases out from (say) February 1st to April 1st.

Expand full comment

With the recent introduction of antivirals and the prospect of vaccine updates, we could benefit a lot from an extra couple of months to prepare.

Expand full comment

Just 2 weeks to flatten the curve...

Expand full comment

Yes. Back in march 2020 there were no boosters or antivirals to ramp up. Things are very different now.

Expand full comment

People panic exactly in the same way...

Expand full comment

It is already everywhere. It is supposed that South Africa is not even the place where it originated. Stopping the flights and closing the borders will not have any effect except will inconvenience people and increase anxiety and further damage mental health.

Expand full comment

Obviously it will have the effect of reducing the number of people with omicron who arrive in the U.S.

Expand full comment

Will it really? The US had for long time prohibited travel from Europe. People who wanted to travel simply travelled through other countries (with even higher infections rates) with mandatory 14 day waiting period. I am not trying to find the logic of this decision.

And related to this. Is at this moment new travellers even the main source of spread? As compared to the ones who are already in the country and becoming superspreaders?

Expand full comment

Of course it will. At the very least, it will delay their entry into the U.S. by 14 days. And of course most people did not go to that trouble. In those 14 days, most people will recover from their omicron infection anyway.

Whether new travelers are the "main source of spread" is irrelevant.

Although I was only refuting your claim that the travel ban would "do nothing" which is obviously false, the percentage of people infected with omicron is far higher in SA than the U.S. so the policy is perfectly sensible if you want to slow the spread of omicron.

Expand full comment

> Whether new travelers are the "main source of spread" is irrelevant.

Why? It is what really matters.

We are not talking about technicalities – if this will reduce something by 0.1% or so – but about practical differences. Most likely it won't make much difference. Or could even make things worse. Especially for poor African countries and whose specialists often need to travel to the US to gain expertise, business deals and academic knowledge, including in matters related to covid. If they are hampered in their efforts, they may fail to contain even worse variants that will spread worldwide.

I am sometimes surprised how insular is American thinking.

Expand full comment

> People who wanted to travel simply travelled through other countries (with even higher infections rates) with mandatory 14 day waiting period. I am not trying to find the logic of this decision.

Passports exist and will tell you the path of someone's travel. If you say "the US didn't actually protect its borders," that is different than saying that travel restrictions don't work.

There is also the nature of exponential growth. If your country only has 100 cases of Omicron in it, keeping out another 300 is skipping ahead two doublings.

The FDA is waiting to approve Paxlovid. We can use the time we buy to approve it and ramp up production, even sending it to SA in exchange for not letting them in here.

Expand full comment

I suppose that currently the US has equal number of omicron infected people than in the countries people are travelling from. I may be wrong but no one really knows and too many assumptions in the past have been wrong that I don't really believe it is worth to believe it this time.

The idea that we just need to hold until Paxlovid is approved is depending on too many unknowns. 1) Is really omicron much more contagious? 2) does it really considerably make vaccines ineffective? 3) Will really Paxlovid work as intended? The costs are biggest factor. Yes, it will be cheaper than monoclonal antibodies but mabs are given only to hospitalized, i.e., 1% of all infected. Paxlovid has to be given to all infected because we don't know who might end in hospital. In total it may turn out much more expensive than mabs and even some people in risk groups will not want to pay and insurance will take too long to sort this out.

At the end I predict there will be no meaningful differences in additional death rates in the US which stopped certain travel and countries that did not introduce such restrictions.

Expand full comment

Is the loss of smell still a common symptom of current strain of covid? It was for the original Wuhan one, so common in the 2020 wave(s). But I do not think it still is, at least for the current Delta. If I am not wrong, Omicron is not really special on this, so it does not give any info about relative severity compared to Delta...

Expand full comment

First time I hear that. So far the loss of smell is considered the defining symptom of covid.

Expand full comment

Not sure where I saw this first... Here is the text i managed to find back: "Notons que le symptôme caractéristique du Covid, la perte d’odorat, qui n’est plus si important avec le variant Delta (responsable en grande majorité des infections au Covid aux Etats-Unis et en Europe actuellement) réapparaît dans le top des symptômes chez les full vaccinés. " it says loss of smell is not common anymore with delta, for unvaccinated and partially vaccinated people (they mean single dose of a two dose vaccine, i guess) . But it is still a common among the fully vaccinated (two shots, or, maybe a shot of Johnson single single dose one). All of this is a little bit strange, they give symptoms depending on vaccination status is to much detail to be believable. Among my friends that were recently contaminated, 0% reported loss of smell... 0 on a total group of 2, so this is super significative ;-p

More seriously, there really is reporting of less loss of smell floating around before Omicron, so even if this is quite shaky, it lower the likeless that lack of loss of smell among Omicron victims is itself significative...

Expand full comment

> While Noah Smith correctly says that restrictions like border closures and other nonpharmaceutical interventions are now useless

I don't understand this talking point. If no one at the border were permitted entry or if everyone who was entering had to quarantine for the duration of their infection, you have very clearly eliminated one infection vector entirely, which can only make your job of containing the spread within a country easier.

Expand full comment

We are not in the position to contain the spread anymore. Why do you think that it is possible to do so in the US or Europe?

Expand full comment

You absolutely can contain the spread to only be *within your own borders*, that's the point. Are you not aware of New Zealand's policy since the pandemic began?

Expand full comment

Even New Zealand hasn't contained the spread. They managed for longer time than other countries but it seems that ultimately they have given up zero covid policy. See https://www.worldometers.info/coronavirus/country/new-zealand/

Expand full comment

There's a difference (sometimes forgotten) between keeping it out entirely and slowing it down.

To keep out Omicron, the US would need to have implemented different policies a few weeks ago. People often say "well, too late now," ignoring the fact that we've done this barbecue several times now, each time told it's too late. Unless Omicron is the last variant[1], we'll be here again, so we should consider those policies going forward.

To slow down, we restrict immigration from any area with noticeably higher rates of infection. The restrictions don't need to be perfect: if we have 100 cases locally we don't want to import 400, but importing 10 is acceptable.

[1] Omicron might be so infectious that it really is the last one, the one that infects everyone, but there is a large bias where everyone believes that history ends with them.

Expand full comment

In theory it could work but in practice it is unlikely that closing borders will make a significant difference in this case. We might have a case that omicron is not that infectious, or those who are already in the country will spread it even more efficiently or the covid spread follows wave like patters with hard-to-predict start and ending times that will happen anyway. Somehow, we have never learned the humility from our failures to model this pandemic.

Expand full comment

I guess the reason that authorities are freaking out about Omicron rather than some other variants is that they have some model of how it will impact human populations based on its genetic structure and the current data on transmission and health. It would be great if they could make this model and assumptions explicit and state what would either confirm or negate this model. Those who disagree could also share their models. Guaranteed we are going to have more scares like this so we need to improve our ability to model their impact based on early evidence otherwise we are going to get Crying Wolf syndrome.

Expand full comment

Science might eventually get to the point when we could look at a virus under a microscope to determine these things, but right now we detect transmissability and lethality the old-fashioned way: by watching the cases.

The longer it takes us to get information on the death rate, the better the news is, because it means deaths are harder to measure because it's less lethal. No news is good news, on this front.

Expand full comment

I think they've been pretty explicit:

This virus seems to have outcompeted Delta in a few locations, which suggests that there's a good chance it will outcompete Delta in others.

The virus has lots of mutations on its spike, which suggests that there's a good chance it will have some amount of immune evasion.

Asking for anything more specific than either of those, like a specific population structure model for how an epidemic unfolds, is demanding the impossible. We don't need to have a specific model for what the Japanese invasion plan is to determine whether or not to react to Pearl Harbor.

Expand full comment

Agree that these factors have been mentioned, but I would like to see an explicit scoring mechanism though that gives a score based on some historical variants performance. You could weight genetic variation, short term growth perhaps and other factors into a numerical score, when it exceeds a certain level the alarm is sounded. Surely we have enough performance data on previous variants by now to be able to create this model. An explicit model has the benefit that it forces coherent logic. It's like writing down your theory of how things work, it clarifies your thinking and makes you aware of gaps in your logic. This actually goes all the way back to the start of the pandemic, we were all arguing over whether covid was significant or not and whether drastic measures were needed. A scoring system could have identified the need for action earlier.

Expand full comment

Do we have past evidence that suggests that Metaculus’s users are good at predicting this sort of thing? This isn’t like a political or sports question where there are lots of people out there who possess different subsets of information, such that if we aggregate the data we’ll get the right answer. There really isn’t a lot of data to go on yet, so mostly this is just a bunch of people guessing, and it’s not clear that Metaculus is frequented by the sorts of people who are likely to have meaningful guesses about this…

Expand full comment

not for covid specifically, but: https://www.metaculus.com/questions/track-record/

Expand full comment

actually you can filter by category. for the covid category the metaculus prediction seems to have an average brier of 0.093.

Expand full comment

Thanks. I guess that’s what I was looking for, though now I’m not sure how to decide which of those earlier questions are really as “guessing about unknowns” as this one is…

Expand full comment

At this early stage in the game, I feel like the prediction markets aren't that useful because we're still making wild guesses from a position of complete ignorance.

What will be interesting is to see how things change over time. In the several days since this was posted, nothing has really changed on the markets, but if we see the odds starting to go towards 0 or 1 then we'll know there's interesting new information coming in.

Or for an alternative view, if you're interested in omicron news you can pretty much ignore all the newspapers and just watch the prediction markets. The headlines right now are full of speculation like "Omicron could be terrible" or "Actually omicron might be really good" or "Omicron might be not terrible not still not good". But until the prediction markets start shifting, you can be confident that all the news coming in is pretty useless.

Expand full comment

Prediction: we will reach the Omega Variant, and people will freak the fuck out because of the name.

Expand full comment

I don't understand how people settled on R0=2.5 for vanilla covid. It seems much higher to me, given the (much more accurately measurable) serial interval of about 5 days, and case doubling time of 3 days (see e.g. US covid deaths by date in March 2020).

Expand full comment

Well, the case doubling rate (1/time) equals the infection doubling rate plus the p(detection|infection) doubling rate, and in places like the US in March 2020 I'd guess the latter was much larger than the former.

Expand full comment

Right. That's why I mentioned *death* statistics and not cases, which are not prone to this error in this context.

Expand full comment

Even so, I'd guess that a considerably smaller fraction of actual COVID deaths were recognized as such and included in statistics in early March 2020 than in late March 2020.

Do we have statistics for total pneumonia deaths in that period.

Expand full comment

You have to look at that graph in log scale in Our World in Data - it's the cleanest increasing exponent I've ever seen in real data. And I believe all candidate deaths in that period were tested for covid, so detection was less or not at all of an issue.

Expand full comment

"all candidate deaths in that period were tested for covid" -- I don't think so, and even if they were only people who tested positive *while alive* were included in the statistic.

Belgium had many more official deaths per capita in the spring of 2020 than most other countries because it was one of the few countries that also counted people who died with obvious COVID symptoms but without a PCR test.

Expand full comment

I don't know that those #'s at Our World in Data are by actual data of death. I think that they are by date reported, perhaps with some adjustment for big data dumps when states highlighted that many deaths reported were from older periods.

OWID references Johns Hopkins CSEE as a source for COVID data. When I look at the Johns Hopkins site, I don't see a clear explanation of whether information on COVID deaths is listed by actual date of death or date reported. For the U.S., however, I think that it's the latter (at least in broad terms), because the death totals for recent days ( e.g., https://coronavirus.jhu.edu/data/state-timeline/new-deaths/new-york/0 and https://www.arcgis.com/apps/dashboards/bda7594740fd40299423467b48e9ecf6 ) are larger than I'd expect to see for actual date of death reporting. That's based on occasions where I've looked at state data sources that provide actual date of death for reported COVID deaths. The last few days are typically at ~0 in these reports due to lag times in reporting of deaths.

Expand full comment

It really disappoints me when otherwise intelligent people are ignorant enough to use twitter.

Expand full comment

I HATE posting on Substack. For the second time in 2 days I have spent upwards of 30 mins. writing a long reply, and Substack suddenly swallowed my text before it was quite complete. Yes, I know I could have avoided this infuriating outcome if I'd typed my reply into a text editor on my computer then copy-pasted into Substack, but why do we have to be communicating on a platform that's so fucking poorly set up that we need to fall back on that sort of workaround?

Expand full comment

Aren't we getting to the point where enough have been infected that we have natural immunity? And don't the vaccines work A LITTLE BIT to prevent spread? Aren't we getting to endemic soon?

Or maybe the new variants also escape natural immunity?

Or maybe not as many have been infected as I had thought.

I don't know.

Expand full comment

Anecdotally, a captured WW2 German communique argued that “it is useless to study the Americans Army manuals, because the Americans don’t read them, and wouldn’t follow them if they did.”

I feel the same way regarding any and all data sets regarding the patterns and behaviors of large groups of people in this day and age. To include data on communicable diseases which you would want “somebody to do something” about. Every single big brained Ivy Leagued nitwit out there has forgotten that human beings are a second order chaotic system. They will change their behavior when they realize they are being observed. A good case study for this is the number of counties in swing states which shifted towards Bush after very concerned UK citizens wrote them letters imploring them to vote for John Kerry.

TL;DR- Put a sock in it with these charts and such. It’s not helpful.

Expand full comment

I really liked this post until the last sentence.

Expand full comment

For some reason doing this with predicttion markets feels a little scuzzy. I suppose it feels too much like "lay, your bets, how many people will die?" Obviously prediction markets could have grear utility but I feel like the way its presented here is a little too cavalier/gleeful.

Expand full comment

Potentially stupid question - is there a reason that there have not yet been any recorded deaths from the Omicron variant? Is this normal with new variants of diseases/is there typically a lag of a few months before deaths start to show up?

I've seen articles like this one:

https://www.bloomberg.com/news/articles/2021-12-08/s-african-weekly-excess-deaths-almost-double-amid-omicron-wave

talking about the increase in excess deaths, but that is not at all a clear indicator that Omicron is running rampant through South Africa - again, mostly because no one who has been diagnosed with it has died according to the WHO.

What is happening here?

Expand full comment