Has Robin ever talked to a doctor or been ill? I need someone to explain to me why this isn't the stupidest thing anyone has ever said. Myself and my girlfriend have both had simple surgeries for life threatening problems, we'd both be dead without medicine.
So I see your point but I do not think you meaningfully interact with what Hanson is saying - in fact it seems like what he brings forward does not effect how you respond to him at all. Even if we can grant that the maximum application of his thesis is just obviously incorrect by no other metric than the successful eradication of certain infectious diseases or things like soft-tissue repair, that does not invalidate everything he has said. That being said, if you are charitable with the person you are responding to, then certainly you should be given pause that for all the praise of modern medicine there are massive problems.
The chemical-imbalance theory of clinical depression has always been controversial and now that the theory is losing support you can only look out into the aftermath of prescribing people so many of these different anti-depressants to see the problems.
How many people were potentially harmed by following the tau theory in Alzheimer's?
How many women have been harmed by the overuse of C-sections for routine pregnancies?
How much medicine has been developed to treat diseases brought about by poor lifestyle choices like obesity or malnourishment from eating processed foods with no nutrients?
Medicine in America is run like a business, and that has some major problematic implications so your response comes off as being dangerously defensive of what may in fact be indefensible.
Cancer is a poor place to look for effects - and especially for a $ per QALY metric which a rough steelmanning of Hanson. Which obviously is orthogonal to a “medicine doesn’t work” argument but might support a “we spend too much money to help the elderly live a few more poor years” generalized argument.
Much better places to look are 1) antibiotics 2) emergency medicine 3) pre-natal and maternity care 4) general surgery, for starters - both on a mortality basis and a QALY basis.
I hate the way this discussion is framed. Robin says "medicine doesn't work." Scott says "medicine does work." Why is the relevant object "medicine?" How does that make any sense? There isn't some one big, monolithic thing called "medicine." There are many different, heterogeneous pathologies, and many different, heterogeneous treatments. My null hypothesis would be some of these are great, and some of them suck (even worse than doing nothing). And having read the entire post, I still think that's exactly the case.
It's easy to believe that modern antibiotics are excellent for treating many bacteria infections. And that some forms surgery, perhaps spinal fusion surgery for degenerative disks, can be worse than doing nothing. And that medicine is a big mixed bag containing everything in between. There's no paradox or puzzle here, and it's just dumb to frame this as a fight between "medicine works" and "medicine doesn't work."
> I think if Robin wants to do something with these insurance study results, he should follow other economists, including the study authors, and argue about whether the marginal unit of insurance is cost-effective - not about whether medication works at all.
As far as I can recall, Robin Hanson generally talks about the marginal unit of healthcare, rather than whether medication works "at all". For example, in his long paper on medical behavior, he talked about "A near-zero marginal health-value of medical care" rather than whether medical care had any value "at all" (see https://mason.gmu.edu/~rhanson/showcare.pdf). I also recall him being clear to say that he's only talking about marginal effectiveness of medicine in his book The Elephant in the Brain.
I agree that the way he talks about this topic can be confusing. But ultimately I suspect that, if Hanson ends up replying to this post, he will say that you misunderstood his views on medicine.
None of this meaningfully disproves Hanson. Hanson's (correctly!) credits spontaneously occuring secular trends with the increase in life expectancy. Others did so before him and there are a bunch of other studies basically coming to the same conclusion, i.e. finding small to irrelevant effects for medical care. This is not new and not surprising, and anybody practicing medicine with open eyes has noticed that the presentations of new cases of practically all diseases seem to ameliorate over time and the clinically large/evident presentations of the past are practically absent in todays practice.
The error here is the same with all the other evidence that has been dragged in to refute this, in that people do not understand that incidence rules health, that comparably large increases in survival in lethal disease will not appreciably change the population's life expectancy when incidence is low, and that incidence has always been low even for those supposedly common deadly diseases of history. Adjuncts to these errors are the failure to understand that most chronic diseases of aging do not result in death or disability if left alone, most treatment fails to meaningfully alter the course of the disease it pretends to treat and the failure to understand lead-time- and length-time-biases, as demonstrated by the diatribe on cancer survival.
I understand the need to believe in medicine when you are working as a physician, but this is sloppy reasoning.
I'm really curious what Robin would say to all the progress on cancer. One might argue that it's a poor use of money, but some of the new cancer treatments have huge effect sizes that would swamp his complaints for high noise.
"I’ve spent fifteen years not responding to this argument, because I worry it would be harsh and annoying to use my platform to beat up on one contrarian who nobody else listens to."
Admittedly I don't have as big of a platform as you do, but one solution I often use for this is to first argue with the contrarian about it behind the scenes, and then only publish the rebuttal if they don't self-correct.
The problem with not getting it corrected at all is that "who nobody else listens to" is wrong. E.g. for a period of time, I found it plausible that Robin Hanson was right, because I assumed he and others had done their basic statistics to check the validity of the claims (especially because I saw e.g. Eliezer Yudkowsky reference the results too). It's only after getting a better mindset for the statistics (many of the problems you mention here show up in *a lot* of studies) that I started disbelieving.
Agreed with this, and I particularly like seeing the improvement in mortality for specific conditions over time like this, it strikes me as a really strong argument for the effectiveness of medicine.
What I do think remains a valid sort-of anti-medicine point is that treatment is vastly less cost-effective than prevention, I recall hearing it being about 50 times less so, and so clearly vast savings could be made through government disease-prevention programs such as dissuasion campaigns against and fees on the externalities of risk factors for disease and especially the broad category of "ingested substances" whether food or recreational drugs and even air pollution; the feeing of externalities ("pigovian taxation") is of course the least econonomically burdensome and indeed in theory if we could properly calculate the value of the externalities it would be economically optimal, since it doesn't require making any government expenditure and to the contrary actually constitutes a source of income for the government and can substitute for an equal amount of economically harmful taxation, so that's what seems to me the most obvious major policy to help reduce healthcare costs, though frustratingly it's foolishly opposed by many and ironically generally the most so by the "taxation is theft" crowd.
So to look on the positive side, as a physician, I feel like I now understand better the perspective of police officers now in response to the Defund the Police movement.
Wow, I wish people would either stop using generic generalizations or read the semantic literature on the variety of incompatible interpretations people make of generics. There is nothing scientifically respectable about summarizing a claim as a generic generalization.
Antibiotics starting with penicillin and moving forward ever since.
Insulin.
Primary PCI for STEMI.
There are many explicit examples where medicine “works”. And when a guy is willing to say a P value of 0.01 is “noise”, there’s really not much point engaging with the guy.
I agree with your distinction btw secondary prevention and primary prevention. Although I think the fundamental difference is in effect size (and NNT).
I think maybe where he “might” have a point (and this is the part that bothers me daily) is in the lack of precision medicine. When a certain treatment has an NNT of 10….that’s a blockbuster….yet in 9 of those people the therapy will provide no benefit, and we have no a priori way of knowing who that 1 lucky person (out of 10) is.
>Come on! Thousands of clinical RCTs show that medicine has an effect. Robin wants to ignore these in favor of insurance experiments that are underpowered to find effects even when they’re there. Then when someone finally does an insurance experiment big and powerful enough to find effects, and it finds the same thing as all the thousands of clinical RCTs, p = 0.01, Robin says maybe we should dismiss it, because p = 0.01 findings are sometimes just “noise”. Aaargh!
You may want to steelman him here. When analyzing large, complex quasi-experimental data, there are many, many analytic choices to make. The more choices to make, the more opportunity to p-hack. As such, an apparent p value of 0.01 is not necessarily an honest p value of 0.01 because there are many ways to turn large than 0.01 values into apparent 0.01. How do we know the authors didn't cheat here? We don't know that. Would need a specification curve analysis or something like that.
Hanson's argument really is just absurd. I cannot imagine that he really believes it. Surely this must be the adoption of an extreme position in order to get attention.
It is possible for it to be simultaneously true that specific medical interventions are beneficial, while the entirety of modern medicine to have no effect, which I believe is closer to Hanson's position than "No medicine works at all".
Bloodletting gets a terrible rap, but we still practice it! It's the standard treatment for hemochromatosis. (Note: If you have hemochromatosis, and you go to a hospital for this treatment, you're both charged for the treatment, and also the blood is generally discarded, at least as of the last time I looked which granted was a few years ago. You can get your treatment for free (or, in some places, get paid for your treatment) by donating blood.)
This, I think, may be a good analogy for Hanson's position: We have a bunch of really effective treatments which are beneficial - and also a lot of treatments that are, on net, negative. If you're just looking into, specifically, blood pressure - you may be looking specifically at the treatments that are beneficial and neglecting the treatments that are harmful.
It's even possible for treatments that are strictly beneficial to be, on net, negative - I think a common example/argument in this direction is that you are more likely to contract a disease if you go to a hospital (as you're more likely to be in proximity to individuals with infectious diseases). Taking that example as a given, there is some level of beneficial hospital treatment where the benefit is precisely offset by that additional risk. This may be particularly obvious or intuitive during a pandemic, when the risk of going to a hospital may be sufficiently high that even very beneficial treatments can be, on net, negative.
Pointing out that individual interventions are beneficial doesn't really argue against the thrust of the claims being made, which necessarily requires extremely large trials like RAND - because the claim isn't that medicine is never beneficial, but rather that the average effect of medicine is not beneficial.
There are two different questions: 1. Does medicine improve health while alive? 2. Does medicine reduce mortality? I believe the second can be shown to be true statistically, but the first one must be examined carefully.
I think one thing worth mentioning about Hanson's overall argument, is that whilst he is skeptical of the effectiveness of medicine wrt health, he is eager to accept other things as improving health such as relating to stress, air pollution, diet, exercise, sleep quality, smoking, excess alcohol consumption etc. and that related to the whole elephant in the brain thing, it is very strange how much attention medicine/healthcare gets politically etc. compared to alternatives, assuming people are actually interested in health.
You have have the patience of Job. Hanson’s claims are so transcendently wrong I could barely read your debunking. I could certainly never even attempt that myself.
To what extent should this lead me to downweight the opinions of Hanson, Caplan, and a bunch of other "nuh-uh things aren't the way you think it is, just look at this one study" contrarians?
It's already reminding me strongly of Caplan's "Case Against Education" -- thing that everyone thinks is important turns out not to look unimportant when measured in some particular way and is therefore useless.
I come to whine about methodology of measuring quality of cancer treatment and nothing else.
We check if someone survived cancer by checking if they were alive 5 years after we diagnosed them with cancer. We also scan more and better for cancer (ignoring even scanning younger people which would limit deaths from other causes), meaning we discover cancers earlier on in their development.
That's of course good for various reasons, but it also means that the same person with the same cancer could be considered a survivor today, but not 50 years ago, even if their cancer progressed and lead to their death in the exact same way, simply because we put 'start' indicator earlier in its development.
It can be corrected for, but you don't indicate you checked for it in any way, so here I am.
So, I think the particular claim being made -- the concept that past-few-decades innovations are fairly unimpressive and medicine is miscredited for general improvements -- is probably true in some fields, but trivially disprovable for all-medicine by one example:
Neonatology.
Neonatal intensive care units are places where reality breaks. A baby can be far sicker and far more premature than you think possible, and be carried out of an NICU by their parents weeks later. In the early 1970s, the leading pediatric textbook wrote that the limit of viability must be about 28 weeks -- the tiniest preemies that could survive. Today, about half of 23-weekers get to grow up. A normal pregnancy is 40 weeks.
There's a measure called the "Apgar scale" routinely used to assess the health of newborns. It's scored from 0 to 10, with 0-2 points available in five different areas (activity, pulse, grimace, appearance, respiration). A score of 0 in one area means nothing -- a baby that doesn't move, or doesn't cry, or doesn't breathe. A baby with a total Apgar score of 0 is in any conventional sense dead.
There are articles with titles like "The long-term outcome in surviving infants with Apgar zero at 10 minutes" and "Improving infant outcome with a 10 min Apgar of 0". These exist because there are so many babies that are born dead, stay dead for ten minutes, and *come back to life*. Extrapolate from there to babies with Apgar 0 at 1 or 5 minutes, which are the usual times when scores are taken. The prognosis for Apgar 0 babies is...not fantastic, but it is not 100% mortality, which it would be by any natural physiological measure. There is at least one baby reported in the medical literature with an Apgar score of 0 at *twenty* minutes who, in a childhood follow-up, was alive and had absolutely no developmental problems of any kind.
This isn't a tradeoff of "they're alive but profoundly disabled", either. Developmental issues are more common in NICU survivors than the general population, but don't track cleanly to such a framework. The definition of "neurodevelopmental impairment" in follow-up studies is rather broad -- amongst other things, it includes literally any level of cerebral palsy, despite the supermajority of people with the commonly-caused-by-prematurity forms of cerebral palsy being able to walk without assistance. The rates of "any 'NDI' including imperceptibly mild cerebral palsy" amongst extreme preemies consistently go down with the death rates; "unimpaired survival" is a metric that improves just as "survival" is. Truly profound disabilities are pretty rare in this population -- a lot of studies and practitioners use the term "severe disability" in ways that don't actually match what parents think of (a recent large study, https://dranniejanvier.com/wp-content/uploads/2023/05/tchildren-10-0088.pdf, found that clinical vignettes of the most common outcomes called "severe NDI" in follow-up studies are rarely considered significant problems by parents).
There's no way neonatology can be called an improvement in non-medicine. 23-weekers -- hell, 22-weekers -- and babies with 10-minute Apgar scores of 0 aren't surviving because sanitation got better. They're surviving because people have pushed up against the limits of reality.
The quotation from the Goldin, Lurie, and McCubbin paper seems to have some strange formatting that causes the "f"s to drop out of "effect". There are three instances of "eect" in the block quotation.
Hanson closes with "A mere 0.1% of U.S. annual medical spending, or $4.2 billion, could fund a far larger experiment, and hopefully settle the matter."
It is a call to reflect on the questions to ask and then separate wheat from chaff.
Yet most of you seem to strawman. "Look at all that wheat".
All three studies only look for the effect of insurance. None of them actively blocked access to medical treatment. An interesting third way would be "health credit". If the non-insured are so good at choosing the right subset of treatments, we can use that to drop two unfavorable cases:
* the patient judges a particular treatment to be beneficial, but cannot afford it right now
* the patient is of indifferent opinion to a particular treatment. But since cost is covered by insurance, he does the doctor a favor and gives in.
I just don't accept the distinction between sanitation and medicine. I understand that we can see sanitation as preventive, and medicine and reactive. But it's the same science that generates both.
And even if he has a point about the benefits to lots of treatments being small on a statistical level, I think that's clearly a different kind of failure mode from blood-letting. Blood-letting literally didn't work; it did nothing to treat (mitigate or cure) the diseases it was used for. Modern drugs have all been checked to see if they do that. It's possible that a subtle health thing is happening, as he suggests, e.g. the overall effect of many modern drugs includes a bunch of side effects that mean the drug is basically a wash, and the apparent gains to life as they are being used are in fact due to a bunch of other factors, like doctors keeping cleaner hospitals... But if all that's true, it's still a qualitatively different kind of failure.
Honestly this just strikes me as a disingenuous take by Hanson from the top. He uses medicine. I strongly suspect Caplan also uses medicine. Medicine is manifestly and obviously useful for treating major injuries and curing directly treatable illnesses through a cause-and-effect mechanism: We can repair broken bones, we can destroy once-fatal illnesses with antibiotics, trauma surgeons save the lives of people who are shot or in car wrecks. All of this is indisputable so it's far more reasonable to conclude "our ways of measuring medical outcomes miss something" than to argue "medicine doesn't work."
"Europeans in 1600 likely prided themselves on the ways in which their “modern” medicine was superior to what “primitives” had to accept. But we today aren’t so sure: seventeenth century medical theory was based on the four humors, and bloodletting was a common treatment. When we look back at those doctors, we think they may well have done more harm than good."
Hanson seems to conflate medieval medical theory and practice. From a historian:
TL;DR: There was a total disconnect between medical theory and practice. Those who practiced medicine on actual patients didn't write about it, and the people who wrote about the centuries-old theory (e.g. the four humors) didn't ever practice. Modern medical theory being bad because the "four humors" theory was bad (as we would understand it today) does not follow.
Some quotes from the article:
"[We] have a doubly difficult time understanding what medieval medicine was, because the people who practiced didn’t write about it, and the people who wrote about medicine thought practice was beneath them."
"This [archeological evidence from leper colony burials] suggests that medieval understanding of these diseases in practice was a lot more sophisticated than theoretical writings suggest."
"There’s only one thing left: leeches and bloodletting, real or no? Real, but not what you think. Bloodletting, leeches, cupping, and cautery were all popular methods of balancing humors, but they were only practiced by the elite. Cupping has been making a comeback in recent years, and leeches never really went away. None of these were done frequently."
I can't really speak to Hanson's other arguments, but drawing parallels to pre-modern medical theory does not help his case.
You quoted some of the caveats in that last study, but I think it's worth emphasizing that their numbers are kind of sketchy. They say
> we estimate that the average per-month effect of the coverage induced by the intervention on two-year mortality was approximately -0.17 percentage points.
But this would give you -4 percentage points over the whole two years, or -2 percentage points in a single year. That's implausibly huge - life expectancy in the US is more than 1/0.02, so a 2 percentage point risk of death is more than your entire annual mortality risk! You could salvage this if the insurance signups were concentrated in really old people, but mostly I just don't buy it. In fact the authors don't buy it either:
> We view the effects at the lower-magnitude end of our confidence interval as most plausible, given the treatment effect magnitudes reported in prior research
They also note that extending the month-level effects linearly might be a bad idea:
> We also present suggestive evidence that the steady-state effect of annual coverage on mortality is less than 12 times our estimated per-month effect due to concavity in the relationship between coverage and mortality
I haven't dived into the statistics for this study enough to independently have opinions about their analysis, but "we found an implausibly huge effect, but our confidence interval extends to the effect size being 6 times smaller than observed, which wouldn't be as implausible, but luckily the confidence interval still misses zero" is not the kind of epistemic state where I feel _great_ about the conclusions.
Scott mentions the leukemia drug imatinib and the lymphoma drug rituximab, both of which I was able to be on (a more powerful successor of imatinib called dasatinib, actually, which I still take daily) and which were the main crux of the regimen which saved my life when I was diagnosed with acute lymphoblastic leukemia in my mid twenties.
My leukemia is driven by a mutation called Philadelphia chromosome only present in a small minority of A.L.L. cases. It was, fun fact, the first cancer mutation ever identified, since it is visible under a powerful microscope by causing some *very* wonky stuff with the 22nd chromosome - they first saw it in 1959.
Previously, leukemias had proven treatable in many cases with the advent of bone marrow transplants: leukemia is bone marrow cancer, so we use intense radiation and chemotherapy to fry your bone marrow into oblivion and then replace it with someone else's who is similar enough to you that you can get by.
But, it didn't really help people with Philadelphia chromosome, which for some reason was much more resistant to treatment. Even a couple of decades ago, other kinds of A.L.L. were seeing 5 year survival rates around 50-60% while diagnosis of Philadelphia chromosome was a death knell; survival rates hovered below 10%. Treatment could usually induce a remission but it almost always came back stronger within weeks or months.
And then, in the early 2000s, a miracle; the development of the first mutation-targeted medication for a cancer, in a weird twist of fate for the first kind of cancer whose mutation was ever identified. Imatinib is a tyrosine kinase inhibitor - a tyrosine kinase being, to my understanding, a kind of enzyme that signals cellular reproduction and which Philadelphia chromosome causes its cells to make a broken version of which is always on, always signaling for reproduction. TKIs find cells that have the broken enzyme and cause cell death. (And also to some amount of regular blood cells too I'm pretty sure since mine makes me continuously anemic and immunocompromised, but I'm not going to look a gift horse in the mouth here, it hits the mark most of the time)
This finally gave patients with Philadelphia positive A.L.L. some hope, as well as patients with chronic leukemias, a much higher proportion of which have Philadelphia mutation origins. People began living longer, and some even began being cured and remain alive today.
There are now two successor generations of TKIs, generation two is dozens of times more powerful than imatinib and generation three is hundreds of times more powerful. Two years ago when I was diagnosed, my doctor gave me a prognosis of not just a 70% chance of 5 year survival, but a 60% chance of reaching a cure - and unlike Philadelphia-negative A.L.L., without even a bone marrow transplant. Outcomes for ph+ and ph- A.L.L. are now more or less similar and it seems possible that in the future, ph+ leukemia could actually be regarded as a favorable mutation for prognosis.
Also part of my treatment regimen (which included more medications than I can count on my hands) was rituximab, because my leukemia also came with the C.D. marker rituximab is able to treat. I'm not too sure on exactly what a C.D. marker is and it seems like neither are most non-hematologist doctors I've asked, but I'm thankful for it, and it improved my chances considerably.
Anyways, cheers to modern medicine! I literally could not get by without it. My leukemia isn't cured - yet, growth mindset - but it's controlled to an extent that it's undetectable on tests most of the time, and if it ever decides to come back I still have several lines of treatment and more powerful TKIs left available to me, even more than I had when I started out.
Minor point but I wouldn’t waste much time with the “patients are just diagnosed younger and therefore do better” line wrt cancer. In many common tumors the cancers typically found in younger people are biologically and clinically different, and often much worse (faster-progressing with a shorter prognosis), than those typically found in older people.
(Breast and lung especially should really be thought of as multiple disparate diseases at this point and imo it makes this kind of broad conversation difficult to illustrate with them, let alone with “cancer” writ large; much better if feasible to zero in on a specific, coherently characterizable tumor type that fits your specific example needs. But I do get that this is a lot to ask in a thirty-thousand-foot-view essay about Medicine Y/N lol.)
I am not sure I agree with Robert here, but I see a few gaps in your arguments.
1. Survival rates. These are heavily dependent on detection timing. Imagine a world where we cannot treat cancer at all, but suddenly discover a technology to detect all cancers 6y earlier. Then the correct comparison would be between 5y survival rates pre-discovery and 11y survival rates post discovery. The cancers we observe post-discovery are much less deadly, on average, than the ones we observed pre-discovery. 5y survival rate would get to a very high level, but it is not clear what the benefit would be as we would be still unable to treat any cancers. Thus, your comparisons of survival rates are biased by better and earlier diagnosis.
2. Bayesian chaining. It is not obvious to me that a claim "medicine does not work" applies only to step 4 of your causal chain. Step 4 corresponds to the "medicines/medication do not work" claim. I think Robert is arguing about "medicine", not "medication", so it is not very nice to switch the goal to "medication" as do in your conclusions.
It is possible that there is a breakdown at steps 2 or 3 in our healthcare as a system that renders it useless while some medicines do work when used properly. I think it is fair to call these step 2 and 3 breakdowns "medicine does not work". For example., imagine a world where doctors make diagnoses completely at random irrespective of the underlying conditions. Then step 2 would be badly broken, and we would not observe good outcomes for the group with greater access to doctors. I think it would be fair to describe this situation as "medicine does not work". (And, yes, I am aware that this hypothetical is not compatible with perfect detection hypothetical in 1)
And don't forget preventative public health measures! These are part of medicine too. Better to not have cholera or dysentery or polio or HBP or excessive LDL (name your condition) in the first place, than to seek treatment after these conditions become a problem. Preventative measures are generally a good thing, and have been driven by biomedical research. Medicine is not only treatment for a preexisting condition: it's also about preventing the problem arise in the first place: and often the interventions are cheap pharmaceuticals. Sun block, statins, polio vaccine, clean water, etc. The counterfactuals here are weird: should we just close all our primary treatment centres and not bother putting in tents, repairing compound fractures, or whatever?
>Even Robin admits this is a real effect; he just classifies it as more physics than medicine.
Does he have the same opinion about really obvious surgery? Clearly everything that ever works is physics in the end, even if we can't discern all the intermediate steps in most cases.
I was raised in Christian Science, a faith-healing cult that avoids all medical intervention (this is an excellent comprehensive documentary on the topic: https://www.youtube.com/watch?v=E7RT4wNhiYQ).
My grandmother died slowly and horribly and pointlessly of a skin cancer which would have been trivially easy to treat (it has a 99% survival rate).
I, and every single surviving family member also raised in Christian Science, now have varying degrees of permanent damage from benign medical neglect.
Had Robin Hanson looked into Christian Science, he would not have formed these conclusions.
I'm surprised the argument centers narrowly on survival. Medicine is also good for improving bodily function and alleviating symptoms.
For example, anti-epileptic medications might be life-saving (I don't know the data), but they certainly help people to live more normal lives. Opioids and benzodiazepines have made the dying process for many people less horrific. Trauma surgery means broken bones can be fixed and a once disabling injury now is only a temporary set-back. There's durable symptomatic control of many chronic diseases with medication (e.g., rheumatoid arthritis, inflammatory bowel disease, SLE, COPD).
I think it's an inevitable epistemic challenge that there are things we don't know now that we'll know in fifty years about the interventions we're currently using - either there are unintended negative consequences or we thought some things were working when they really weren't (e.g., cancer drugs approved via the accelerated pathway end up showing no benefit when tested more rigorously). That's just the nature of how science evolves over time. In the face of this epistemic challenge, though, is it better to do nothing or something? I think that's the question best asked on a disease-by-disease, person-by-person basis. Will this intervention help this person right now? I spend a lot of my time in palliative care helping folks to navigate just those kinds of questions.
FYI Hanson & Caplan are not alone, this skeptical position is also the main point of Jacob Stegenga, Medical Nihilism (https://www.amazon.fr/Medical-Nihilism-Jacob-Stegenga/dp/0198747047) : "This book argues that if we consider the ubiquity of small effect sizes in medicine, the extent of misleading evidence in medical research, the thin theoretical basis of many interventions, and the malleability of empirical methods, and if we employ our best inductive framework, then our confidence in medical interventions ought to be low"
I do not agree Medicine is not effective, but I think his point is more about what dominates in terms of increasing lifespan and the sort of cost-benefit analysis. For example, it could be that all the cancer, cardiovascular etc medicine we have only increases lifespan on aggregate by like 2 years (because people who have these diseases are likely to die of some other affliction soon enough.) There was a paper arguing that curing *all* cancer would lead to an increase in lifespan of ~ 3 years. I think it's fair to ask: is it worth spending so much money for such small benefits or could this money be used better? I do not think these questions matter a lot because people will still prefer getting medicine, but it's a fair question and I do think we might overestimate the average add in lifespan from medicine beyond antibiotics and stuff like this.
That being said, to me this is the best argument to invest in ageing research.
Clearly reasoned (absence of evidence is not evidence of absence), but are you tilting at windmills? Hanson's perspective seems preposterous. Can we seriously question that diphtheria vaccine has no effect? that antibiotics for bacterial pneumonia are worthless? that insulin is useless for type 1 diabetics? that antihypertensives have no impact on malignant hypertension?
The other angle to do this is doing it bottom-up. Surely dialysis and kidney transplants work against diabetes? Surely getting your broken arm put in a cast works? Surely antibiotics work? Surely having surgeons treat that stab wound works? Surely having your infected appendix removed works? Surely the Covid vaccine worked? Surely antihistamines work? And so on and so on - once you've established that vast slews of medical interventions *obviously* work, then any odd statistics that claim they don't must surely have methodological errors.
As a MD and a researcher setting up clinical trials, I found Robin Hanson considerations absurd and obsolete. It is like discussing if the earth is flat or at the center of the solar system. I think there are far more relevant thought experiment to discuss such AI doctors, AI as a government to avoid needless wars, robot assisted society, space colonization etc.
The Scott Alexander answer is excellent. Vaccines, antibiotics, antidiabetics, antiarythmics, immunotherapy, robot surgery have demonstrated their clinical efficacy in terms of survival rates, quality of life, autonomy and mobility etc.
Okay, can anybody explain to me why Robin Hanson is taken seriously about anything? From my view, the one thing he's done is write some SF about emulated humans in a computer. But that seems to make him a Respected Big Thinker.
The guy comes out with "yeah I had my broken arm fixed, but are we *really* sure medicine is any good?"
Now, do we have contemporary theories that will eventually turn out to be like the four humours? Probably, but that will be due to the advancement of knowledge like everything else. Next time Hanson breaks his arm, do you think he'll stay at home and do some deep breathing, because medicine is still on the level of Galen, or will he head to the local hospital?
The only sense I can make out of all this is that he enjoys being a feckin' eejit, I mean contrarian, and taking the "if everyone jumped off a cliff, would you do it do? yes, mom, I would!" position. That's fun, but do it consistently and you don't come across as a gadfly, you seem to be a silly person.
What's next - "is food actually necessary? history shows primitive peoples believed you could acquire the characteristics of a creature by consuming its flesh, hence they would eat the heart of a lion to become brave, we know today this is nonsense, but Big Agriculture and Big Dietician Industry has convinced us we need to 'eat' to live, I propose instead that all you need is air".
I’m glad to see you pushing back on this claim as it may undermine the urgency for much more investment into medical research. I would like to see 10x the amount of medical research being done. And why not? There is a lot of room for improvement when it comes to reducing the suffering from physical and mental health problems.
You can increase medicine use with no effects on questionnaires or measured health outcomes. When I was younger, I was very allergic to tuna and salmon. I went and reintroduced them into my alimentation. It took a lot of time, probably shot me to the top 5% of medicine use for my age and country, but did not change a lot on how I would have responded or scored on most questionnaires. I was very good a avoiding my allergies, so I doubt my adverse event rate or death rate moved any direction. I was also not very stressed about the allergy or the procedure and would have not scored differently on mood or stress. You could build a case that nothing changed and that it was useless with the kind of metrics tracked by those studies. Except from the fact that I can eat tuna and salmon now, and that while it's probably not the most efficient use of medicine, it still works and I am happy with it.
How quickly we forget! Does he even know that the mid-thirties were once called "the heart attack years"? I saw a huge change in heart disease over my forty years in practice, partly due to less smoking, but also partly due to diet and statins. Another example: when I was diagnosed with CLL, I looked up the stats and they were dismal. Fortunately things have changed, with targeted treatments the outlook is vastly better, and I may even be cured by an HSCT.
Thank you for writing this! I know you said no one takes Hanson and Caplan seriously, but I definitely take them seriously, even I do not always agree with them. I work in healthcare, and though intuitively I thought Hanson was wrong, I could not figure out exactly where his mistake was. This left me with a gnawing feeling that I might just be biased and everything I do for my job is pointless and doesn’t help anyone. This article reduces that feeling.
> This might seem like a silly question: in Europe of the seventeenth century, the average lifespan was in the low 30s. Now it’s the low 80s. Isn’t that difference due to medicine? In fact, the consensus is now that historical lifespan gains are better explained by nutrition, sanitation, and wealth.
Wow. Hanson is committing one of the best-known statistical errors here. First off, it doesn't pass the basic "smell test." (Does anyone *really* believe that 300 years ago we had a bunch of elderly people in their 30s running around?!?) I can't help but wonder, does Hanson not realize just how well-known of a mistake this specific claim is? It's practically a textbook example of wrongly grouping together unlike data.
"Historical lifespan gains" are virtually nonexistent. Modern conditions have improved things a little bit at the margin, but certainly not by decades! That "average" is explained almost entirely by throwing a whole bunch of 0s, 1s, and 2s into the data that skew the overall number far too far to the low side, and then we came up with vaccines and other treatments that did away with infant mortality almost entirely.
Once you separate infant mortality from adult mortality, a very different picture emerges: the ancient figure of 70-80 years for a human lifespan (Psalm 90:10) has remained almost completely unchanged for millennia, and we're just barely starting to improve on it today.
I think saying “no” to “does modern medicine work?” encompasses multitudes, the possible answers being (strongest no to weakest, clickbaity no):
1. It’s all fake and pointless, all medicine may as well be sugar pills, surgery’s just a more drastic alternative to acupuncture.
2. A few bits and pieces work (eg antibiotics) but the same’s true of witch doctor potions and Chinese medicine; most of it’s pointless.
3. Quite a few of the pills work; most are overrated and some are rubbish. Doctors themselves are a pointless bureaucratic hurdle to pill-buying who add no value.
4. Modern medicine has some kind of paradigm/worldview which is better than chance but fundamentally wrong (a bit like Cartesian vertices).
6. Health insurance achieves nothing, in spite of the pills working.
7. Health insurance is overrated.
Hanson’s position seems to be 2, but his studies only get him to 6 at best. The study you’d need for 2 is a comparison of effect sizes (or a better measure of efficacy) against placebo for a panel of medicines weighted by how commonly they’re prescribed.
It might be too difficult, especially with easily available data, but maybe some extra stats could make a stronger case from a lot of these studies -- if the main problem is a largeish number of marginal results.
(I'm not sure how much further effort it's worth going to on this topic but I think the below is generally interesting and worth trying.)
Karnataka measured over 80 outcomes and there are various others.
Unlike published RCTs maybe we can hope they don't suffer from selection (publication) bias, after all they seem to have published a lot of non-results.
Hopefully they also published anything that looked like the medicine was harmful!
Case 1 (no real effect): We'd expect to see a range of results, with no particular bias towards positive outcomes and some marginal results in both directions.
Case 2 (beneficial effect of medicine): We'd expect to see a range of results but with a positive drift. It might be hard to produce well powered formal test, though there are some simple things one could do:
a. Plot effect estimates with CIs, maybe there's a visually obvious positive bias
b. A sign test: What fraction of findings suggest a positive effect vs. negative.
@Scott: If you have or know where the get the data relatively easily, I'm happy to poke around, make some plots and try some tests. Maybe with some kind of pre-registration of what to try, once the basic parameters of the data are clear?
Can the correct medical treatment work if administered properly? Yes, for many diseases the effect is massively positive.
Is going to the hospital and doing what the doctors tell you to likely to increase your health? This is where Robin has the chance to be correct. He identifies a huge gap between what we think medical intervention can do (massively increase our length and quality of life) and what it actually does in practice (very specific and often marginal improvements in a subset of the ailments humans have). This gap creates a huge mismatch in expectations which is extremely dangerous. Patients are massively biased towards intervention. They get frustrated with the modest gains of real medicine and will seek more and more intervention to try to get the increases in health they imagine medicine should be able to give them. Intervention is dangerous. Doctors are likewise frustrated with the mismatch between their very high social status and their very modest ability to heal. They will go along with or even encourage the unnecessary interventions. Every surgery has risk, every medication has side effects. If people seek intervention far out of proportion to real benefits, then the iatrogenic effects of treatment can cancel our the small positive effects of medicine. In this way, even if medical treatment is real, the total effect of the medical sector can plausibly be negative or within statistical error of zero.
Okay, to steelman the argument: without antibiotics and the advent of techniques such as x-rays, 21st century medicine would be as ineffective as 17th century medicine, where high mortality rates were due in part to deaths from infections that could not be treated either prophylactically, as with inoculations and vaccinations, or concurrent with the advent of the infection.
Lack of effective painkillers and anaesthesia was also contributory, as well as lack of knowledge about dietary requirements and deficiencies, basic hygiene theory, and knowledge of organ function.
Since they did not yet have advanced understanding, they propounded theories to explain illness and disease which we know now to be grossly mistaken. Similarly, in future, there may be theories which we today assume to be correct or at least to have explanatory power, but further advances will reveal to be mistaken.
That being the best I can say for Hanson's argument, even in the 17th century they did have *some* effective treatments. Medicine was based as much on a practical and pragmatic view as on theoretical; you might not know exactly how foxglove or willowbark worked to cure, but you could see that dosages of these were effective:
"Were any important discoveries made in the 17th century that helped improve doctors knowledge?
In the 1620s an Englishman named William Harvey, who had studied at the great Italian medical school in Padua, discovered that blood circulates around the body, the heart acting as a pump with valves to control the flow. King Charles I encouraged Harvey’s efforts after seeing his work. King Charles II, who came to the throne in 1660 after the death of Cromwell, was also interested in everything scientific, including medicine.
In 1661, a chemist called Robert Boyle published a book called The Sceptical Chemist, which described how the body takes in something from the air to breathe. Boyle also established that without this important gas, which we now know as oxygen, animals and birds would die. In 1662, Charles II granted a Royal Charter to the Royal Society and this encouraged scientists to attempt new experiments. However, despite such promising developments, many superstitions were still accepted as truths in the 17th century."
> after you’ve already gotten the cancer, so it’s hard to see how nutrition, sanitation, etc could explain this
If a person gets cancer or a heart attack, it seems like an obvious hypothesis that they might improve their nutrition, exercise, etc.
The fact that this hypothesis isn't even considered in this article is a great microcosm of the overconfidence of modern scientists.
And I'm guessing there isn't great data on it, but it's fascinating how biases rear their heads on all sides of an argument, and most people don't see it.
It seems to me that while so many people appeal to science, they use poor quality science to rationalize biases and intuitions, rather than saying, "Hey, the science here sucks. Let's bring everyone together from all sides and design a great study to evaluate." COVID is a perfect example: the world was ready to run RCTs on a massive scale and all we got was tiny, crappy ones and decisions mostly based on non-RCTs.
I believe that the workings of healthy bodies aren't studied nearly enough. Unless I've missed something, the result is treatments usually drugs) aimed at relieving symptoms, but the treatments have serious side effects, and sometimes weird side effects that aren't related to the ailment in any obvious way.
There are billions of people who don't have schizophrenia and don't have dry mouth, either. How does this work?
I realize it could be hard to get financing for what seems like abstract knowledge that isn't closely related to a specific illness or a cure for it, but abstract knowledge can end up being very important.
I have no strong opinion about how much of medicine as now practiced is worth it, though I would be on more than Hansen says and less than people who trust the whole system think.
I'm sleepy, so I might just be being stupid, but this doesn't seem 100% coherent at first glance:
> "In Robin’s model, these extraordinary studies would have to be bias or chance, and totally coincidentally at the same time somehow better nutrition made leukemia patients (but not uterine cancer patients) twice as likely to survive."
If we grant that the study/medicine is somehow bullshit, then surely we're granting that the survival rates you're referencing here are bullshit data? Or maybe I'm misunderstanding the association between this data point and the research in question. Like I said, I'm sleepy.
Fwiw I’ve read a lot of what Hanson wrote about this, and my impression was always that his framing was “medicine doesn’t clearly work in aggregate”, instead of “no individual type of medicine works”. My understanding might be right or I might be implicitly taking a saner-but-wrong interpretation.
In any case, I think there is a very good question here: if we assume medicine really, obviously works, e.g. by using arguments in this article (and tbc I do), then why can’t we support this with population-wide sociological studies? We spend a very significant fraction of our total wealth on medicine, and it should distress us that we can’t (or simply don’t try hard to) find a high-level aggregate effect.
"Medicine never works" is sure catchier than "some medical interventions don't work very well," but it is only the latter than has a chance of being plausibly defended. When I was in my late 30s, I ruptured a disc in my back at L5-S1. This was excruciatingly painful, to the point that even sitting was close to impossible. I had to lie on the waiting room floor prior to surgery. I had a very successful laminectomy, without which I would be walking with a cane and taking opioids every day of my life. If this happens to Hanson, what's his plan? If he really believes medicine never works, then he might as well go to a witch doctor, or "traditional Chinese medicine," or do nothing at all. Get a cane and some oxycodones, I guess.
Without taking an overall position on the debate (yet), a couple of points:
"[This paper] finds that “plan effects on blood pressure” were three times higher for hypertensives for non-hypertensives; that is, unlike statistical flukes (which we would expect to affect everyone equally), the effect was concentrated in the people we would expect doctors to treat."
This seems mistaken to me -- conditional on the assumption that blood pressure is the 1 of 30 that coincidentally had positive effects, high effects on hypertensives specifically is something we should *expect* to find.
"Third, you need the diagnoses to result in more treatment (eg blood pressure medication)."
An additional factor that could be a separate step of the funnel or could be lumped into this third step is medication compliance, which we know is often poor.
Generally when I'm making the argument that medicine doesn't work, I look to country data. Time series life improvements don't really impress me when those same improvements show up everywhere in the world regardless of how much medical spending those countries are doing.
Spending a short amount of time looking up some numbers, the US spends some thirteen thousand dollars per capita on healthcare. Ecuador spends around five hundred dollars per capita. Life expectancy at 15 is 80 in both countries.
Reducing the rate of mortality by 0.06% is implausibly high, at least if I'm understanding it right. Preventing 1 out of ~1,600 people from dying is a plausible result from providing those 1,600 health insurance, but that's not what happened. Only about 1% of people who were sent a letter actually got insurance, so that means that about 1 out of *16* people who got insurance had their life saved in the next two years as a result of getting insurance. I don't believe that in the slightest.
I personally know intelligent, educated people who profess views similar to Robin's (so it's more than just two contrarians). Sometimes this is just a basic failure of reasoning: only sick people go to the doctor, so if you never go to the doctor, you'll never get sick. But often, I think it's some form of mood affiliation or virtue signaling: medicine is artificial, unnatural, "Western"; natural, "Eastern", or "native" solutions must be better.
Robin's own stated explanation for increased life expectancy contradicts his argument, anyway. Sanitation is good because unsanitary conditions make people sick. Improving sanitation is straightforwardly a form of preventative medicine.
> Robin’s argument (that medicine doesn’t work) assumes that the only possible failure is at step 4, and that the failure must be a true failure rather than one of statistical significance. But in fact there are failures at every step (although I kind of have to stretch it for step 3), and the authors of the papers tell us explicitly that these are most likely failures of statistical power.
Is that his argument? I think if doctors randomly assigned diagnoses to patients (i.e. complete failure at step 2), Robin's overall argument would be correct, since he's talking about the medical system as a whole, not just medicine.
I wish Hanson wouldn't insist on making the most insupportable version of the argument. It's a waste of everyone's time. If we were debating the medical and cost effectiveness of specific treatments, that would be great. We should do that. Let's not have the college dorm room stoner version of the debate.
Isn't healthcare very likely one of the main mechanisms wealth actually leads to better health (especially if we're already pulling out nutrition and sanitation as separate mechanisms)?
Maternal mortality would seem to be another line where one could argue that medicine works. Especially potent since it's one of the cases where medicine was very likely a net negative (in the 1700's and 1800's) and has now moved to be a definite positive.
Also traumatic injury, hasn't treatment of, for example gunshot victims, improved survivability enough that many of the contrarian folks (though perhaps not exactly these contrarian folks) have hypothesized that it messes some with the crime statistics?
AIDS medications? AIDS went from a death sentence to a manageable condition with targeted therapies. See also curing hepatitis-c.
Really medicine is such a broad category that, as others have noted, it seems just crazy to treat it as one thing. There are undoubtedly parts of it that 'work'. Treatment of infections diseases (vaccines, antibiotics, targeted anti-virals) and traumatic injury/event treatment would seem to be exceptionally clear cases. Of course there are parts that don't work or are even negative to health, and in some cases we have a pretty good idea what those are, in others it's less clear. And the claim for health insurance working is more nuanced (particularly for the marginal increase thereof, and the specific implementation may make substantial difference). And many parts scream out of better cost/benefit (and not just monetary, consider over screening and associated unnecessary treatment side-effects that are argued around, for ex. breast and prostate cancer). But just saying it doesn't work is nuts.
Feels that it is somewhat fair to say that the marginal expense side of medicine has lost efficiency. But, that is a banal statement that relies on how effective our groundwork of medical treatments even is today? Stated differently, I would expect that research dollars spent on medical treatments will not go as far today; because they first have to cover known advances that we are building upon.
Taking that to a stronger statement that medical treatments don't work is markedly absurd. Just look through a family tree going back a hundred years. Be sure to ask how many kids aren't included because they died before they were 5. Then pay attention to how many died of sepsis in the past.
I love Hanson. One of the amazing things about him is that, despite his self-identifying as not very good at social things, he appears to be a natural at some of the Dark Arts. This is a perfect example: by the time you're arguing about specific claims you've already lost. You might as well be arguing with the sportscar salesman about what kind of shirt you'll look coolest in driving this convertible. If medicine were as great as some think it, nobody would have to get down into the weeds to discuss at what ages people are getting cancer.
I think Robin would probably agree that a drug works. Not all of them but sure, they exist.
The real question is what is the outcome of all medical interventions as typically practiced by the system? If there’s a common drug that helps leukemia and a common surgery that kills more cvd patients then it helps, how do things actually net out? How could you possibly know?
Probably medical system helps but the big takeaway for me is do we really know? Put this in the category of other questions like Caplans do we know education helps kids?
I come away from this thinking A) at least some medicines and condition-screening "works", and B) providing free health care is a bad deal for taxpayers when the recipients are under 45. The rational health policy this would lead to is one where Medicaid expansion is limited to older people, and to the extent certain types of coverage are mandated for insurance carriers it would be heavily focused on things like annual screenings for cancer and hypertension.
While some of this is reflected in PPACA coverage minimums, it also forces insurance companies to provide care that the majority of "customers" will never use, notably substance abuse counseling, pharmaceutical interventions for addiction, expensive patented pre-exposure drugs for unmarried/promiscuous gay males to avoid HIV, and a range of birth control products. And as a result, we have deductibles that aren't far off from the $5K ones RAND considered "very bad" in the 70s, back when insurance was mostly seen as a way to cover accidents and emergencies rather than an overall care plan.
It also continues to baffle me that the people who seemed the most politically active in advocating for various "Medicare 4 All" style programs in the past decade were the younger Millenials in their 20s and 30s, statistically the people for whom increased access to care does demonstrably nothing and who are decades away from any of the mandatory screenings that have benefits. It leaves me to wonder whether these people are particularly neurotic and overuse health care access, particularly sickly, or whether their immediate concern was for older family members. The people most likely to agitate for student loan relief are people who owe a lot of student loans, but the people most likely to agitate for health care access are the people who don't need it?
I've always understood Hanson's considered claim as "we have a huge overemphasis on medicine" not that "modern medicine doesn't work." That's why he suggested cutting medical spending by half, not entirely. This piece doesn't rebut that claim.
I get the sense that Robin Hanson is trolling people for the lolz.
When I was ten years old, I developed abdominal pain. My mom took me to the doctor, who palpated my abdomen and announced that my appendix was about to burst and I needed surgery immediately.
The post-op recovery sucked really badly, especially for a child (I couldn't laugh, because it hurt so much; I had to eat gross pureed carrot soup and wasn't allowed any fried foods or chocolate for months; I hate carrot soup to this day). But the surgery SAVED MY LIFE. Yes, yes, n = 1 anecdote blah blah, but this particular anecdote is MY LIFE. I'd like to see how Europeans in the 1600s, with their leeches and four humors, would have taken care of my appendix.
Robin Hanson can take his contrarianism and shove it where the sun don't shine.
It seems like the dodge of attributing longer lifespans to “nutrition, sanitation, and wealth” is dumb? Wealth is the dumbest since it manifests as better nutrition and better sanitation (and access to medicine, but we can ignore that for now), so it’s redundant. But even nutrition and sanitation are sus since….aren’t those kind of medicine? Like I guess one could say medicine is only drugs, but is it? Why have nutrition and sanitation gotten better if not for “medical” research?
To me the explanation is obvious. The way insurance works, or is intended to work, is that people rarely need healthcare but would struggle to afford it when they are the rare person who needs it. If a particular condition affects 1/1,000 people and costs $1,000 dollars to fix, then insurance charges everyone $1 and pays for the people who need it. At least in theory and discounting administrative costs, etc.
There are some problems with this simplistic model. The chain of events mentioned in the article is the source of most of it. You can't get people treatment if you don't know they have the disease, which you can't know if they aren't getting to the doctor, which they can't do if they can't afford it/don't have insurance. So instead of 1/1,000 of the population getting treatment, you have a much much higher percentage getting checked for the condition, some proportion getting a more significant follow-up check to confirm, and then some final number getting treated. This final number is likely higher than 1/1,000 because it includes people who are mistakenly diagnosed or who fall below some threshold where treatment is necessary but still get the treatment anyway (which the patient will often demand and medical malpractice makes difficult to say no to).
So the costs that should be a certain figure are significantly higher, and most of the difference is not helping actual patient outcomes. You're still only benefitting 1/1,000 of the population, while spending resources on most of the population.
If he's taking the strong stance that all medicine is bunk, then I strongly disagree. If he's taking the weaker stance that a significant portion, likely an overwhelming majority, of our expense on healthcare is wasted/ineffective, I do agree with that. In fact, I think that explains why we get such small results over the population despite knowing that medical interventions *have* to be working in many cases.
Roughly speaking, I would guess that the first 1% of GDP put into healthcare is likely miraculous in results, the next few percent are very effective, the next few percent are actuarily measurable, the next few are a wash in a cost/benefit sense, and the last few are actually making things worse. Averaged out over the whole cost, we end up being unable to link spending with outcomes in a measurable way.
I think his claims totally make sense if you think about the almost 20% of GDP spent compared to how difficult it is to measure positive outcomes. The problem is not that medicine is worthless/ineffective, but that we spend a majority of our healthcare money on things that even in theory can't help people feel better (diagnostics, real estate, administration) to find the small minority of cases that we can help with relatively cheap medical care (relative to the total cost).
The reason Robin interprets null results as negative results is that studies do not fall from the sky (except natural experiments). The RAND study was designed to detect a certain level of effect. Its failure to detect that effect is a statistically significant rejection of the hypothesis of effectiveness held by the designers. This is a dangerous line of thought, but also having a prior on a non-quantitative hypothesis like "medicine is real" is also a dangerous line of thought. The zero point isn't magic. Medicine has obvious health costs. If marginal medicine is designed to have a 10:1 health benefit to health cost and then it is discovered to only have 1/10th the benefit it was believed to, it is quite easy for it to be net negative.
Lots of problems with this analysis on both sides. Imagine a trial where you give out free aspirin to people and track how their headaches improve. The biggest finding is that people didn't have to go out and buy their own aspirin as much. Okay, so this study wasn't about whether people got aspirin (giving it away for free isn't the same as preventing people from getting it) so much as the financial benefit of having to pay for something or not.
I'm mostly NOT sympathetic to Hanson's point, but the decrease in depression and reduced financial burden might be correlated with "didn't have to pay for health insurance" in the Oregon study. Let's take a charitable middle ground argument and say that at least 15% of health care spending obviously works (especially a lot of interventional stuff people in the comments have mentioned; I'd also add cancer therapies with good overall survival benefits, like imatinib, but probably not most modern medicines).
On the other side, let's say at least 50% of health care spending is just ineffective cope for "your lifestyle choices made you unhealthy and we're going to pretend to fix all that". We'll set aside the other 35%, for now, as 'uncertain benefit'. If we cut health care spending from 20% of GDP to 10% of GDP, would people be better off?
(Would people be better off if we shifted 10% of GDP to helping people get fit instead? Probably not. Increased nutrition advice seems to be inversely correlated to people getting thin and fit. Either it doesn't work or it harms people. We need good evidence on this one before we roll it out widely.)
The assessments of whether health care make you better off or not are terrible. That increase in HTN meds? First, I would assume the most severe cases start off on the meds, so increasing HTN medication availability should yield diminishing returns. But what were those returns? Lower blood pressure. Really?! Who cares whether their BP is 140/90 vs. 137/87? Patients care about whether they get headaches, or SOB, or more importantly whether they're going to die of a heart attack. It's death and pain that people get hung up on.
Doc: Where does it hurt?
Patient: In the HbA1c.
Nobody does this. People don't care about their hemoglobin's post-translational modifications, the salinity of their urine, or how much triglycerides are in their blood. They care about living long, better lives. So the real question is: do people live better and/or longer with the intervention. Most of these studies don't measure that over the long term, and aren't designed to. Therefore we can't conclude one way or the other about efficacy.
The correct study is to look at either lifespan or overall survival over long time horizons. There's a reason nobody is doing this. That study is impractical and by the time you get the results it's been a half century and medicine has changed. There's no exception that says "bad data can be substituted for good data because it's impractical to do it right". The exception is, "if we can't do it right, then we just don't know."
I'd be interested in his analysis of addiction treatment. One argument would be that after a brief medical detox, treatment does nothing to increase the odds of sobriety. The other argument would be that people who want to remain sober need multiple interventions across many domains such as economic, housing, emotional, social, and legal.
I'm stopping near the beginning to comment. My spouse is a highly cited, known medical researcher of integrity, and he has been saying the exact. same. things. for years. Our educated friends dismiss him out of hand in spite of hard data and extreme expertise. He thinks statins made a difference in quality of life late in life, and they meet the criteria for validated research. He likes my hip replacement; surgery matters. Antibiotics in moderation matter.
But in general, much medicine and especially socialized medicine is an elaborate grift at the top, supported by well-meaning people who do not question simplistic, conventional narrative.
There are a some things in medicine that work. Vaccines. Anti-hypertensives. Statins. ACE inhibitors. Pap smears. Post-MI care. There are a million studies showing disease-specific as well as all-cause mortality benefits to those interventions. (And some studies that challenge those, but the preponderance of evidence is in favor of mortality reduction.) But there are a million devices, medicines, tests, and woowoo for which we either have no evidence or evidence of no, and which we let the medical system sell with impunity. Population health improvements have been striking. Covid barely made a dent in what we have been able to achieve over the past 200-300 years. Some of that is clearly nutrition, sanitation, etc. And some medical technologies clearly help spin the wheels underneath those mortality reductions but beyond the obvious suspects, we do not know what else is helping. Just some ballpark numbers: out of about 1 million devices, medicines, and tests, maybe 1000 are clearly effective. That's 0.1% with clear benefits. That's enough to account for the majority of the ~2-3mo improvement in life expectancy yearly. But it's also not enough for someone to claim with pretty good certainly that >99% of the stuff in medicine is probably nothing (and I include placebo in nothing).
I largely agree with your conclusions. People are far too quick to assume that more expensive insurance leads to better treatment. Medical insurance is actually very disconnected from health outcomes, as you show in your funnel. That is why expanding health insurance yields tepid results.
I think the problem is not that modern medicine does not work. It is largely about diminishing returns per marginal dollar invested, and the fact that there are other factors that are far more closely liked to health outcomes.
Genetics and healthy habits do far more for long-term health than medical treatments and insurance.
It will save a lot of effort for readers of this blog to recall that Scott is not pedantic and therefore the phrase "claims that medicine does not work" is an extremely valid rhetorical description of the fact that Hanson is arguing that the marginal value of health care is literally zero - even dismissing the Obamacare study with significant statistical power
Furthermore if you do find the 'marginal health spending is not relatively valuable' argument to be persuasive and wish to defend it you should probably support the concept that the argument should be specifically made in a more defensible and persuasive manner!
I own the book and flipped to the relevant chapter, and the section about modern medicine opens with this sentence, "Medicine today is different in one crucial regard: it's often very effective."
I suspect he will just say that he doesn't believe that medicine doesn't work and that this is a strawman.
I don't have the book in PDF, so I'm going to great pains to write these quotes, but here's more.
"The big historical improvements in medical technology don't tell us much about the value of the marginal medicine we consume in developed countries. Remember we're not asking whether some medicine is better than no medicine, but whether spending $7000 in a year is better for our health than spending $5000. It's perfectly consistent to believe that modern medicine performs miracles and that we frequently overtreat ourselves."
Do most people think that higher life expectancies are due to medicine treatments? I don't think most 70 year olds have had a life saving medical procedure during their lives.
A lot of money in medicine is spent doing stuff "just in case" (like unnecessary tests, MRI's, check ups) as well as making the experience more pleasant for the patient. You could design a maximally cheap health care system from the top down where patients can't choose their physician, can't see a specialist without fulfilling specific guidelines, don't get access to any examinations that aren't evidence based, where there are long waiting periods for everything that isn't urgent, -- and this would save a lot of money probably without statistical detriment to outcomes. Patients would hate it though. (This is what public health care is like in some countries.) A lot of people with enough money or private insurance would still spend a lot of money to get more "luxurious" care (like going to a specialist right away or getting an MRI even when there is no strict medical need.) The point is that only part of the money in medicine goes to medical outcomes, per se.
I mean, if European countries that spend about 10% of their GDP on health care get about the same outcomes as the United States that spends nearly 20% (and has a higher GDP per capita), it is pretty clear that theoretically the United States could save a lot on healthcare. You don't need to try to argue that "modern medicine doesn't work any better than blood letting".
Trephination is still done occasionally today. When a person gets a concussion, the brain swells inside the cranium. Trephination relives the pressure and minimizes brain damage. No doubt there were many, many, head traumas in early human evolution, so it seems reasonable that the procedure was thought out and its usefulness deduced. I forgot to mention that trephination is known in early human history all over the world, so it's unlikely it was local religious myth of some sort. References available in my Fundamentals of Cognitive Science (Routledge).
The quote starting with "The rate of mortality among previously uninsured 45-64 year-olds" seems to have had some copy/paste errors, there are missing letters and extra spaces inserted.
I'm dying from recurrent / metastatic squamous cell carcinoma of the head and neck (R / M HNSCC_frustrating), so the sections about cancer in particular stand out to me. Regardless of the current state-of-the-art for cancer treatment, personalized and mRNA vaccines are likely on the verge of revolutionizing cancer treatment.
Take the HNSCC that's killing me: I got a partial glossectomy in Oct. 2022. Mine had some high-risk features, but I was assured that, with radiation therapy, it wouldn't recur In retrospect, I obviously should've done chemo and radiation, but at the time I was pleased to not need chemo, and I foolishly didn't look deeply into the data on recurrence, which is common for HNSCC, and I didn't seek second opinions.
Docs are reluctant to impose systemic chemo because of the side effects. But Transgene has a personalized vaccine that is supposed to prevent HNSCC recurrence: https://www.nec.com/en/press/202304/global_20230418_01.html: "In the head and neck cancer trial to date, all patients treated with TG4050 have remained disease-free, despite unfavorable systemic immunity and tumor micro-environment before treatment," And most of these personalized vaccines have essentially no side effects.
Moderna's mRNA-4157 platform also looks good: https://jakeseliger.com/2024/04/12/moderna-mrna-4157-v90-news-for-head-and-neck-cancer-patients-like-me/, not only in R / M HNSCC, but in melanoma and lung, too. Right now mRNA-4157 is only being tested in the recurrent / metastatic setting, as far as I know, but the logical time to use it is probably when initial surgeries are done: cut the cancer, sequence it, and then vaccine against it to prevent recurrence.
Right now, from a society-wide perspective, the healthcare I've been getting probably fails the cost-benefit test (apart from the fact that the data I'm generating for clinical trials helps move the state-of-the-art forward). My quality of life is low, and while treatment has been extending my life, it almost certainly won't lead to remission. And even if a clinical-trial drug somehow leads to complete remission, I'll never be able to sleep or speak normally again (https://jakeseliger.com/2023/08/27/on-being-ready-to-die-and-yet-also-now-being-able-to-swallow-slurries-including-ice-cream/). A few months ago my brother casually referred to me being disabled, and I was momentarily confused: Who was he talking about? But he was in fact right: I'm disabled and unlikely to ever be able to think or work in the way I did before losing my tongue.
But that should change! Part of the reason I'm so frustrated by the FDA is that mRNA-4157 and TG4050 should already be available for HNSCC. Instead, they're stuck in trial hell, while HNSCC patients like me suffer recurrences and then die.
I remember the 2008 Oregon experiment. I assume that the researchers had to throw something together very quickly if in order to take advantage of the opportunity to perform a controlled study with random assignment. That's why they ended up with self assessment questionaires that the researchers acknowledged might not reflect actual improvements in health, and measures of clinical results where the improvements were too small to be statistically significant
After the study was released, David Brooks claimed on the PBS New Hour that it showed that people didn't benefit from Medicaid--but I note that Brooks never announced that he was giving up <em>his</em> health insurance.
US health spending as a percentage of GDP has been just over 17% every single year since 2008 except for two specific years: 2020 and 2021. Those two years obviously were impacted by COVID on both sides of the ratio, and for 2022 the percentage settled right back down to 17.3%.
My point here is not about whether 17.3 compared to 19.7 (the 2020 peak pct) invalidates Hanson's argument. Rather it is that his cherry-picking of an obvious outlier year, and then rounding it up rhetorically to represent the norm ("we spend 20% of GDP on medicine"), is enough to me to conclude that he's not presenting any actual serious argument he's just internetting.
Scott concludes: "I think if Robin wants to do something with these insurance study results, he should follow other economists, including the study authors, and argue about whether the marginal unit of insurance is cost-effective"
...fair enough, but you seem to forget what the product insurance (all types of insurance) provides/is meant to provide.
The product is not better health, or a promise of better health.
The product is peace of mind. The product is the knowledge that if you need health care, in particular expensive care, it will not bring economic ruin to yourself or your family.
This is the reason health insurance is usually popular, even in population segments that are seldom in need of health care.
If health insurance should turn out also to improve health outcomes, that is great. But if so it is an added bonus. It is not the point of insurance, public as well as private.
Maybe he doesn't consider insulin a medication because it's technically a hormone. Maybe he excludes *all* life-saving and life-extending treatments and medical procedures.
If he did this, there may be a compelling economic argument. But that's not the argument he's making.
Modern RCT trials are allowed to assume side effects are exactly zero if the P value of side effects is <0.05 between the control and treatment arms. Because of the base rates of other diseases, this can lead to treatments passing RCT that are neutral or even negative on net.
Most treatments have very high "number to treat", being in a hospital is dangerous because of chance for MRSA, surgical errors, etc. drugs studies get to ignore side effects unless they are really obvious, often they can get away with garbage proxy metrics as has been discussed by Scott often this blog.
RFK Jr (yes, I take him AND Bryan Caplan seriously) makes an interesting point in his little-noted most-recent book, The Wuhan Cover-Up, that vaccine research may kill as many people through lab leaks as they do, or might, save through effects of the resultant vaccines. Think COVID-19, though there are MANY others, obviously of smaller scale. The lab leaks are, indeed, VERY common, and potent, too.
Then, of course, there are capers like Anthony Fauci's with AZT, and so on.
There is a massive difference between "medicine doesn't help" and "lowering the cost of medicine doesn't lead to large jumps in health outcomes."
For my everyday life, my use of medicine is bottlenecked by inclination and time (as I have no pressing medical issues). For the majority of emergencies, I am similarly lucky enough to be able to afford healthcare. It's only that when the medical issues is very rare or the benefits slim that my insurance is likely not to cover me.
>It would be easy to round Hanson’s position off to something weaker, like “extra health care isn’t valuable on the margin”. This is how most people interpret the studies he cites.
>I’ve spent fifteen years not responding to this argument, because I worry it would be harsh and annoying to use my platform to beat up on one contrarian who nobody else listens to. But I recently learned Bryan Caplan also takes this seriously. Beating up on two contrarians who nobody else listens to is a great use of a platform!
Does Caplan really think medicine is generally useless, or does he merely espouse the mostly uncontroversial position that "extra health care isn’t valuable on the margin?"
All he writes is that:
>I’m puzzled. The idea that the quality of health care has improved is already almost universally accepted. Economists who argue that healthcare expenditures are wasteful are self-consciously challenging this standard view, highlighting surprisingly strong evidence that marginal medicine fails to improve health. See Robin Hanson’s “Cut Medicine in Half” for details.
Not only does he reference that marginal benefit of medicine, his actual point is almost the opposite; that the conventional wisdom is that the quality of health care has improved. In that context, although he makes passing reference to Hanson, it isn't even clear that he espouses the weaker fairly uncontroversial version of the arguement. If, e.g. he thought there were a 25% chance that (the weak version of) Hanson's argument were correct, that could still qualify as "surprisingly strong evidence," given that he notes that the position challenges the standard view.
Hanson's point is about whether medicine works. Rather than add in the complexities of insurance coverage (RAND, Oregon and Karnataka) to refute his point, you could probably have just said "Look at the UK, Canada and other countries where healthcare is (effectively) free - or at least where the population doesn't worry about massive medical debt (contra the US studies) and knows how the, possibly new to them, system works (contra the Indian study) and lo and behold, health outcomes have indeed gone up MASSIVELY in the last century."
I mean, the article would have been shorter and more boring, but my first thought when I read Hanson was "Using the US isn't helping your case."
This isn't original to me, but I think it's worth flagging: five-year survival rates after diagnosis can be massively improved just by diagnosing earlier, even if you don't do anything to meaningfully improve cancer survival, assuming increasing lethality over time. I'm not saying that we've done nothing to make cancer safer, but I'd prefer a chart that looked at X years of survival after a Stage 3 diagnosis (or a more closely matched comparison).
I'm sure someone else has invoked the famous Orwell quote elsewhere in this comment section; but in case they haven't - "some ideas are so absurd that only an intellectual could believe them".
There's a trap that contrarians often fall into, of seeing the justification of outlandish propositions as some sort of cognitive challenge. Something like, "proving black is white is really hard, but I reckon I could pull it off." At some point such people are so caught up in their own cleverness that they lose track of whether what they're defending is actually true or not.
I’m generally very sympathetic to your side of the argument. However, if I was Hanson’s lawyer, exhibit A might be the opioid epidemic. It can arguably be traced back to the over prescription of opioids, and has had a massive negative impact on health in the US, an impact that wouldn’t show up in a health insurance RCT, because the epidemic eventually spread throughout society. Maybe medicine has or could create other issues like this (more resilient diseases?)? As counsel I would also discuss the direct evidence for medical care causing deaths through accidents, infections, etc. I don’t think these things would net out in Hanson’s favor, but would be interested to see you address them.
I appreciate you going into this topic. Hanson's position has always seemed both solid and extremely challenging to fit into any consistent model of the world.
(It would also be great to have a more general post on "How much evidence should we require before giving credence to '[entire area of study] just doesn't work' theories?")
I am biased in Scott's favor. But I was a bit disappointed that his argument seemed to proceed as if Hanson claims that there have been no improvements in medical outcomes over time. The question is not how much things have improved, but what caused it, which I think is harder to say. Maybe some of those studies address this, and it seemed too obvious for Scott to hammer on it (or I skimmed over his hammering). I have been surprised by things before, so I wish I thought this really settled the issue. If it did, I missed it.
I find it odd that economists - the pseudoscience to beat the bands - produces people who not just attack other more scientific fields (although medicine isn’t a very hard science) and prove their lack of statistical ability while doing so.
Being myself a physician, I tend to underplay medicine's success, but even I see that the main thing here is perspective. There's a bias to see anything that transitions from "cutting edge high tech" to "low tech" as obvious and commonplace, as "low hanging fruit" in retrospect. as something that was always there.
In the logarithmic graph of medical success, nothing will ever beat the 4 main things that brought us to where we are today - Sanitation, Obstetrics, Vaccination and Antibiotics - all are now so low tech so as to seem divorced from medicine, but they did originate as medicine technology. We can add exercise and nutrition as additional ones today.
The rest of medicine suffers from availability bias - if you don't know anyone close to you that has had a heart attack, stroke, cancer, physical trauma such as a car crash, appendicitis, need for an organ transplant, one of the plethora of autoimmune diseases, etc., it is quite reasonable to think the whole field is useless. But I think that's the hallmark of a functioning system, conceptually - it's invisible to you until you need it, and then it provides.
I'll join the chorus of voices saying that both Hanson's opinion and Scott's refutation are underspecified in exactly what phenomenon they are evaluating.
But consider some specific cases: not just obvious things like antibiotics for infection, but on one hand, take the random person whose debilitating ailment is alleviated by a very specific medicine. If you know older people you can probably name one. Then consider the other random person who takes a daily regime of a dozen pills prescribed by a variety of doctors, who can barely manage to figure out which pills they have already taken or not, and whose liver or kidneys or brain are slowly being intoxicated by the combination. My quick take is that #1 is beneficial, and #2 is harmful ("iatrogenic" is the word nowadays). But you can't easily have one without (some of) the other. By hypothesis #1's medicine is highly specific, so if you under-invest in medical care, patient #1 is out of luck and might as well kill themselves. But #2 is basically a question of *quality*; what they need is a good doctor to review their entire regime and rationalize it. So the balance points towards "we want better medicine" rather than less of it.
The other thing, if you take the broadest possible view, is that the entire modern-industrial-capitalist environment is making us sick and then turning around and selling us dangerous remedies for it. I can sympathize with this view. Barbara Ehrenreich (look her up!) explains it at length in her book Natural Causes. But at this kind of level it's hard to go from global diagnosis to practical applications. And it's not like pre-industrial people were known for their longevity and great health anyway.
It seems like something went wrong copying this quote or it's just shown incorrectly in my browser (firefox 124.01) .
> Using treatment group assignment as an instrument for coverage, we estimate that the average per-month effect of the coverage induced by the intervention on two-year mortality was approximately -0.17 percentage points. We caution, however, that the magnitude of the mortality eect is imprecisely estimated; our con dence interval is consistent with both moderate and large eects of coverage on mortality. At the same time, our estimated con dence interval is su ciently precise to rule out per-month eects smaller in magnitude than -0.03 percentage points, including the estimate from the OLS regression of mortality on coverage across individuals.
Seems like the fs got replaced everywhere except at the first effect where there also is an additional weird character.
Copying it directly from the paper "con dence" I get the same result. However, the text is rendered "correctly" in the pdf.
> I’ve spent fifteen years not responding to this argument, because I worry it would be harsh and annoying to use my platform to beat up on one contrarian who nobody else listens to. But I recently learned Bryan Caplan also takes this seriously. Beating up on two contrarians who nobody else listens to is a great use of a platform!
I'm out of the loop here. Is this intended to be playful ribbing? Because it sounds a little mean. I'm aware Hanson is a proto-LessWronger but not really aware of who Caplan is.
One problem with the funnel (especially for marginal care) is that at some of the steps, there's some probability that the effect will be bad. I go to the doctor more often when the doctor can't help me, and one of those times I catch the flu from one of the other patients. The attempts at diagnosis do me more harm than good--I have an allergic reaction to the contrast agent for the CT scan, for example. I get put on an additional drug for a marginal thing (the doctor gives me antibiotics for my cold to placate me) and have an allergic reaction to them.
I have no idea how big this effect is, but I'm sure it exists. And we should expect to see it have its biggest impact when we're going to the doctor for marginal stuff. If I suddenly go numb on one side of my body, my worries about medical error making me worse off are *way* smaller than my worries that I'm having a stroke; if I have an annoying cold and go to the doctor for it, the probability he's going to do me harm is higher than the very low probability he's going to do me any good.
I think you are arguing a straw man version of Robin Hanson's view. He does not claim that all medicine is ineffective. His claim is that the helpful treatments are offset in the aggregate by harmful interventions. I refer to the latter as "Hansonian medicine." His explanation for observations that greater access to health insurance does not improve health is that it increases access to harmful interventions as well as to beneficial ones.
Interesting post! I think the claim that all medicine doesn't work is too strong but you should check out the book "Medical Nihilism" for a nice Bayesian argument against efficacy of most drugs. Tldr: there are like 30 drugs doing all the work while the rest are free riding on that reputation.
I'm a HUGE fan of Robin Hanson, and for that reason I'm all the more thankful for Scott tackling the parts of his writing where Robin seems obviously egregiously contrarian to the point of being just wrong.
Thank you, Scott! Please engage Robin in the future, as well.
I cracked laughing out loud at the following:
"I’ll follow Robin’s lead in dismissing the entire medical literature - every RCT of every medication or treatment ever published - because it might have “huge biases,” and try to rely on other sources."
My mom eats like a saint and takes her blood pressure medication and she still struggles. No, she's not secretly cheating or something, I know how she eats. I have heard a lot of stories like this. Maybe the proven isn't medicine but blood pressure medicine?
I think the antihypertensive point furthers Robin's argument to a certain degree. Last time I checked, while lowering blood pressure is associated with lower all-cause mortality, the effect is pretty minimal. Don't get me wrong, there is a pretty significant *relative* effect, but the *absolute* reduction is minimal.
I think this distinction is what Robin's getting at most of the time. He's never really arguing medicine doesn't work, it's more like the cost of it outweighs the utility of it most of the time because our great treatments are not so great at extending lifespan in absolute terms.
Against the strongest versions of the claim, the obvious smallpox and polio; and antibiotics are good arguments too.
Has Robin ever talked to a doctor or been ill? I need someone to explain to me why this isn't the stupidest thing anyone has ever said. Myself and my girlfriend have both had simple surgeries for life threatening problems, we'd both be dead without medicine.
So I see your point but I do not think you meaningfully interact with what Hanson is saying - in fact it seems like what he brings forward does not effect how you respond to him at all. Even if we can grant that the maximum application of his thesis is just obviously incorrect by no other metric than the successful eradication of certain infectious diseases or things like soft-tissue repair, that does not invalidate everything he has said. That being said, if you are charitable with the person you are responding to, then certainly you should be given pause that for all the praise of modern medicine there are massive problems.
The chemical-imbalance theory of clinical depression has always been controversial and now that the theory is losing support you can only look out into the aftermath of prescribing people so many of these different anti-depressants to see the problems.
How many people were potentially harmed by following the tau theory in Alzheimer's?
How many women have been harmed by the overuse of C-sections for routine pregnancies?
How much medicine has been developed to treat diseases brought about by poor lifestyle choices like obesity or malnourishment from eating processed foods with no nutrients?
Medicine in America is run like a business, and that has some major problematic implications so your response comes off as being dangerously defensive of what may in fact be indefensible.
Your Caplan link doesn't work, should be https://betonit.substack.com/p/reflections-on-goff-and-the-cost
Cancer is a poor place to look for effects - and especially for a $ per QALY metric which a rough steelmanning of Hanson. Which obviously is orthogonal to a “medicine doesn’t work” argument but might support a “we spend too much money to help the elderly live a few more poor years” generalized argument.
Much better places to look are 1) antibiotics 2) emergency medicine 3) pre-natal and maternity care 4) general surgery, for starters - both on a mortality basis and a QALY basis.
I hate the way this discussion is framed. Robin says "medicine doesn't work." Scott says "medicine does work." Why is the relevant object "medicine?" How does that make any sense? There isn't some one big, monolithic thing called "medicine." There are many different, heterogeneous pathologies, and many different, heterogeneous treatments. My null hypothesis would be some of these are great, and some of them suck (even worse than doing nothing). And having read the entire post, I still think that's exactly the case.
It's easy to believe that modern antibiotics are excellent for treating many bacteria infections. And that some forms surgery, perhaps spinal fusion surgery for degenerative disks, can be worse than doing nothing. And that medicine is a big mixed bag containing everything in between. There's no paradox or puzzle here, and it's just dumb to frame this as a fight between "medicine works" and "medicine doesn't work."
> I think if Robin wants to do something with these insurance study results, he should follow other economists, including the study authors, and argue about whether the marginal unit of insurance is cost-effective - not about whether medication works at all.
As far as I can recall, Robin Hanson generally talks about the marginal unit of healthcare, rather than whether medication works "at all". For example, in his long paper on medical behavior, he talked about "A near-zero marginal health-value of medical care" rather than whether medical care had any value "at all" (see https://mason.gmu.edu/~rhanson/showcare.pdf). I also recall him being clear to say that he's only talking about marginal effectiveness of medicine in his book The Elephant in the Brain.
I agree that the way he talks about this topic can be confusing. But ultimately I suspect that, if Hanson ends up replying to this post, he will say that you misunderstood his views on medicine.
None of this meaningfully disproves Hanson. Hanson's (correctly!) credits spontaneously occuring secular trends with the increase in life expectancy. Others did so before him and there are a bunch of other studies basically coming to the same conclusion, i.e. finding small to irrelevant effects for medical care. This is not new and not surprising, and anybody practicing medicine with open eyes has noticed that the presentations of new cases of practically all diseases seem to ameliorate over time and the clinically large/evident presentations of the past are practically absent in todays practice.
The error here is the same with all the other evidence that has been dragged in to refute this, in that people do not understand that incidence rules health, that comparably large increases in survival in lethal disease will not appreciably change the population's life expectancy when incidence is low, and that incidence has always been low even for those supposedly common deadly diseases of history. Adjuncts to these errors are the failure to understand that most chronic diseases of aging do not result in death or disability if left alone, most treatment fails to meaningfully alter the course of the disease it pretends to treat and the failure to understand lead-time- and length-time-biases, as demonstrated by the diatribe on cancer survival.
I understand the need to believe in medicine when you are working as a physician, but this is sloppy reasoning.
I'm really curious what Robin would say to all the progress on cancer. One might argue that it's a poor use of money, but some of the new cancer treatments have huge effect sizes that would swamp his complaints for high noise.
"I’ve spent fifteen years not responding to this argument, because I worry it would be harsh and annoying to use my platform to beat up on one contrarian who nobody else listens to."
Admittedly I don't have as big of a platform as you do, but one solution I often use for this is to first argue with the contrarian about it behind the scenes, and then only publish the rebuttal if they don't self-correct.
The problem with not getting it corrected at all is that "who nobody else listens to" is wrong. E.g. for a period of time, I found it plausible that Robin Hanson was right, because I assumed he and others had done their basic statistics to check the validity of the claims (especially because I saw e.g. Eliezer Yudkowsky reference the results too). It's only after getting a better mindset for the statistics (many of the problems you mention here show up in *a lot* of studies) that I started disbelieving.
I don't know why you can't find the book. It's here: https://annas-archive.org/md5/be2a9bdb7087cc9b26659211f3cf8447
Agreed with this, and I particularly like seeing the improvement in mortality for specific conditions over time like this, it strikes me as a really strong argument for the effectiveness of medicine.
What I do think remains a valid sort-of anti-medicine point is that treatment is vastly less cost-effective than prevention, I recall hearing it being about 50 times less so, and so clearly vast savings could be made through government disease-prevention programs such as dissuasion campaigns against and fees on the externalities of risk factors for disease and especially the broad category of "ingested substances" whether food or recreational drugs and even air pollution; the feeing of externalities ("pigovian taxation") is of course the least econonomically burdensome and indeed in theory if we could properly calculate the value of the externalities it would be economically optimal, since it doesn't require making any government expenditure and to the contrary actually constitutes a source of income for the government and can substitute for an equal amount of economically harmful taxation, so that's what seems to me the most obvious major policy to help reduce healthcare costs, though frustratingly it's foolishly opposed by many and ironically generally the most so by the "taxation is theft" crowd.
So to look on the positive side, as a physician, I feel like I now understand better the perspective of police officers now in response to the Defund the Police movement.
Wow, I wish people would either stop using generic generalizations or read the semantic literature on the variety of incompatible interpretations people make of generics. There is nothing scientifically respectable about summarizing a claim as a generic generalization.
Antibiotics starting with penicillin and moving forward ever since.
Insulin.
Primary PCI for STEMI.
There are many explicit examples where medicine “works”. And when a guy is willing to say a P value of 0.01 is “noise”, there’s really not much point engaging with the guy.
I agree with your distinction btw secondary prevention and primary prevention. Although I think the fundamental difference is in effect size (and NNT).
I think maybe where he “might” have a point (and this is the part that bothers me daily) is in the lack of precision medicine. When a certain treatment has an NNT of 10….that’s a blockbuster….yet in 9 of those people the therapy will provide no benefit, and we have no a priori way of knowing who that 1 lucky person (out of 10) is.
Haven't spend much time researching this but other than Caplan I think Random Critical Analysis (https://randomcriticalanalysis.com/2016/11/06/us-life-expectancy-is-below-naive-expectations-mostly-because-it-economically-outperforms/) and Gwern (https://gwern.net/drug-heuristic#sn16) also seem sympathetic to Hanson's view.
>Come on! Thousands of clinical RCTs show that medicine has an effect. Robin wants to ignore these in favor of insurance experiments that are underpowered to find effects even when they’re there. Then when someone finally does an insurance experiment big and powerful enough to find effects, and it finds the same thing as all the thousands of clinical RCTs, p = 0.01, Robin says maybe we should dismiss it, because p = 0.01 findings are sometimes just “noise”. Aaargh!
You may want to steelman him here. When analyzing large, complex quasi-experimental data, there are many, many analytic choices to make. The more choices to make, the more opportunity to p-hack. As such, an apparent p value of 0.01 is not necessarily an honest p value of 0.01 because there are many ways to turn large than 0.01 values into apparent 0.01. How do we know the authors didn't cheat here? We don't know that. Would need a specification curve analysis or something like that.
Hanson's argument really is just absurd. I cannot imagine that he really believes it. Surely this must be the adoption of an extreme position in order to get attention.
It is possible for it to be simultaneously true that specific medical interventions are beneficial, while the entirety of modern medicine to have no effect, which I believe is closer to Hanson's position than "No medicine works at all".
Bloodletting gets a terrible rap, but we still practice it! It's the standard treatment for hemochromatosis. (Note: If you have hemochromatosis, and you go to a hospital for this treatment, you're both charged for the treatment, and also the blood is generally discarded, at least as of the last time I looked which granted was a few years ago. You can get your treatment for free (or, in some places, get paid for your treatment) by donating blood.)
This, I think, may be a good analogy for Hanson's position: We have a bunch of really effective treatments which are beneficial - and also a lot of treatments that are, on net, negative. If you're just looking into, specifically, blood pressure - you may be looking specifically at the treatments that are beneficial and neglecting the treatments that are harmful.
It's even possible for treatments that are strictly beneficial to be, on net, negative - I think a common example/argument in this direction is that you are more likely to contract a disease if you go to a hospital (as you're more likely to be in proximity to individuals with infectious diseases). Taking that example as a given, there is some level of beneficial hospital treatment where the benefit is precisely offset by that additional risk. This may be particularly obvious or intuitive during a pandemic, when the risk of going to a hospital may be sufficiently high that even very beneficial treatments can be, on net, negative.
Pointing out that individual interventions are beneficial doesn't really argue against the thrust of the claims being made, which necessarily requires extremely large trials like RAND - because the claim isn't that medicine is never beneficial, but rather that the average effect of medicine is not beneficial.
There are two different questions: 1. Does medicine improve health while alive? 2. Does medicine reduce mortality? I believe the second can be shown to be true statistically, but the first one must be examined carefully.
A few typos:
"For example, in the Oregon study, the insured group used about 33% more health care than the insured group."
Copying from the pdf for the Goldin, Lurie, and McCubbin paper left the "ff" glyphs out of your paste.
I think one thing worth mentioning about Hanson's overall argument, is that whilst he is skeptical of the effectiveness of medicine wrt health, he is eager to accept other things as improving health such as relating to stress, air pollution, diet, exercise, sleep quality, smoking, excess alcohol consumption etc. and that related to the whole elephant in the brain thing, it is very strange how much attention medicine/healthcare gets politically etc. compared to alternatives, assuming people are actually interested in health.
You have have the patience of Job. Hanson’s claims are so transcendently wrong I could barely read your debunking. I could certainly never even attempt that myself.
To what extent should this lead me to downweight the opinions of Hanson, Caplan, and a bunch of other "nuh-uh things aren't the way you think it is, just look at this one study" contrarians?
It's already reminding me strongly of Caplan's "Case Against Education" -- thing that everyone thinks is important turns out not to look unimportant when measured in some particular way and is therefore useless.
I come to whine about methodology of measuring quality of cancer treatment and nothing else.
We check if someone survived cancer by checking if they were alive 5 years after we diagnosed them with cancer. We also scan more and better for cancer (ignoring even scanning younger people which would limit deaths from other causes), meaning we discover cancers earlier on in their development.
That's of course good for various reasons, but it also means that the same person with the same cancer could be considered a survivor today, but not 50 years ago, even if their cancer progressed and lead to their death in the exact same way, simply because we put 'start' indicator earlier in its development.
It can be corrected for, but you don't indicate you checked for it in any way, so here I am.
So, I think the particular claim being made -- the concept that past-few-decades innovations are fairly unimpressive and medicine is miscredited for general improvements -- is probably true in some fields, but trivially disprovable for all-medicine by one example:
Neonatology.
Neonatal intensive care units are places where reality breaks. A baby can be far sicker and far more premature than you think possible, and be carried out of an NICU by their parents weeks later. In the early 1970s, the leading pediatric textbook wrote that the limit of viability must be about 28 weeks -- the tiniest preemies that could survive. Today, about half of 23-weekers get to grow up. A normal pregnancy is 40 weeks.
There's a measure called the "Apgar scale" routinely used to assess the health of newborns. It's scored from 0 to 10, with 0-2 points available in five different areas (activity, pulse, grimace, appearance, respiration). A score of 0 in one area means nothing -- a baby that doesn't move, or doesn't cry, or doesn't breathe. A baby with a total Apgar score of 0 is in any conventional sense dead.
There are articles with titles like "The long-term outcome in surviving infants with Apgar zero at 10 minutes" and "Improving infant outcome with a 10 min Apgar of 0". These exist because there are so many babies that are born dead, stay dead for ten minutes, and *come back to life*. Extrapolate from there to babies with Apgar 0 at 1 or 5 minutes, which are the usual times when scores are taken. The prognosis for Apgar 0 babies is...not fantastic, but it is not 100% mortality, which it would be by any natural physiological measure. There is at least one baby reported in the medical literature with an Apgar score of 0 at *twenty* minutes who, in a childhood follow-up, was alive and had absolutely no developmental problems of any kind.
This isn't a tradeoff of "they're alive but profoundly disabled", either. Developmental issues are more common in NICU survivors than the general population, but don't track cleanly to such a framework. The definition of "neurodevelopmental impairment" in follow-up studies is rather broad -- amongst other things, it includes literally any level of cerebral palsy, despite the supermajority of people with the commonly-caused-by-prematurity forms of cerebral palsy being able to walk without assistance. The rates of "any 'NDI' including imperceptibly mild cerebral palsy" amongst extreme preemies consistently go down with the death rates; "unimpaired survival" is a metric that improves just as "survival" is. Truly profound disabilities are pretty rare in this population -- a lot of studies and practitioners use the term "severe disability" in ways that don't actually match what parents think of (a recent large study, https://dranniejanvier.com/wp-content/uploads/2023/05/tchildren-10-0088.pdf, found that clinical vignettes of the most common outcomes called "severe NDI" in follow-up studies are rarely considered significant problems by parents).
There's no way neonatology can be called an improvement in non-medicine. 23-weekers -- hell, 22-weekers -- and babies with 10-minute Apgar scores of 0 aren't surviving because sanitation got better. They're surviving because people have pushed up against the limits of reality.
The quotation from the Goldin, Lurie, and McCubbin paper seems to have some strange formatting that causes the "f"s to drop out of "effect". There are three instances of "eect" in the block quotation.
Hanson closes with "A mere 0.1% of U.S. annual medical spending, or $4.2 billion, could fund a far larger experiment, and hopefully settle the matter."
It is a call to reflect on the questions to ask and then separate wheat from chaff.
Yet most of you seem to strawman. "Look at all that wheat".
All three studies only look for the effect of insurance. None of them actively blocked access to medical treatment. An interesting third way would be "health credit". If the non-insured are so good at choosing the right subset of treatments, we can use that to drop two unfavorable cases:
* the patient judges a particular treatment to be beneficial, but cannot afford it right now
* the patient is of indifferent opinion to a particular treatment. But since cost is covered by insurance, he does the doctor a favor and gives in.
I just don't accept the distinction between sanitation and medicine. I understand that we can see sanitation as preventive, and medicine and reactive. But it's the same science that generates both.
And even if he has a point about the benefits to lots of treatments being small on a statistical level, I think that's clearly a different kind of failure mode from blood-letting. Blood-letting literally didn't work; it did nothing to treat (mitigate or cure) the diseases it was used for. Modern drugs have all been checked to see if they do that. It's possible that a subtle health thing is happening, as he suggests, e.g. the overall effect of many modern drugs includes a bunch of side effects that mean the drug is basically a wash, and the apparent gains to life as they are being used are in fact due to a bunch of other factors, like doctors keeping cleaner hospitals... But if all that's true, it's still a qualitatively different kind of failure.
Honestly this just strikes me as a disingenuous take by Hanson from the top. He uses medicine. I strongly suspect Caplan also uses medicine. Medicine is manifestly and obviously useful for treating major injuries and curing directly treatable illnesses through a cause-and-effect mechanism: We can repair broken bones, we can destroy once-fatal illnesses with antibiotics, trauma surgeons save the lives of people who are shot or in car wrecks. All of this is indisputable so it's far more reasonable to conclude "our ways of measuring medical outcomes miss something" than to argue "medicine doesn't work."
"Europeans in 1600 likely prided themselves on the ways in which their “modern” medicine was superior to what “primitives” had to accept. But we today aren’t so sure: seventeenth century medical theory was based on the four humors, and bloodletting was a common treatment. When we look back at those doctors, we think they may well have done more harm than good."
Hanson seems to conflate medieval medical theory and practice. From a historian:
https://acoup.blog/2021/05/20/meet-a-historian-robin-s-reich-on-making-sense-of-medieval-medicine-humors-weird-animal-parts-and-experiential-knowledge/
TL;DR: There was a total disconnect between medical theory and practice. Those who practiced medicine on actual patients didn't write about it, and the people who wrote about the centuries-old theory (e.g. the four humors) didn't ever practice. Modern medical theory being bad because the "four humors" theory was bad (as we would understand it today) does not follow.
Some quotes from the article:
"[We] have a doubly difficult time understanding what medieval medicine was, because the people who practiced didn’t write about it, and the people who wrote about medicine thought practice was beneath them."
"This [archeological evidence from leper colony burials] suggests that medieval understanding of these diseases in practice was a lot more sophisticated than theoretical writings suggest."
"There’s only one thing left: leeches and bloodletting, real or no? Real, but not what you think. Bloodletting, leeches, cupping, and cautery were all popular methods of balancing humors, but they were only practiced by the elite. Cupping has been making a comeback in recent years, and leeches never really went away. None of these were done frequently."
I can't really speak to Hanson's other arguments, but drawing parallels to pre-modern medical theory does not help his case.
You quoted some of the caveats in that last study, but I think it's worth emphasizing that their numbers are kind of sketchy. They say
> we estimate that the average per-month effect of the coverage induced by the intervention on two-year mortality was approximately -0.17 percentage points.
But this would give you -4 percentage points over the whole two years, or -2 percentage points in a single year. That's implausibly huge - life expectancy in the US is more than 1/0.02, so a 2 percentage point risk of death is more than your entire annual mortality risk! You could salvage this if the insurance signups were concentrated in really old people, but mostly I just don't buy it. In fact the authors don't buy it either:
> We view the effects at the lower-magnitude end of our confidence interval as most plausible, given the treatment effect magnitudes reported in prior research
They also note that extending the month-level effects linearly might be a bad idea:
> We also present suggestive evidence that the steady-state effect of annual coverage on mortality is less than 12 times our estimated per-month effect due to concavity in the relationship between coverage and mortality
I haven't dived into the statistics for this study enough to independently have opinions about their analysis, but "we found an implausibly huge effect, but our confidence interval extends to the effect size being 6 times smaller than observed, which wouldn't be as implausible, but luckily the confidence interval still misses zero" is not the kind of epistemic state where I feel _great_ about the conclusions.
As in the usual Bayesian meme, "I will bet you $10 sun doesn't explode", or in this case "I will bet you $10 that your arm surgery worked"
Scott mentions the leukemia drug imatinib and the lymphoma drug rituximab, both of which I was able to be on (a more powerful successor of imatinib called dasatinib, actually, which I still take daily) and which were the main crux of the regimen which saved my life when I was diagnosed with acute lymphoblastic leukemia in my mid twenties.
My leukemia is driven by a mutation called Philadelphia chromosome only present in a small minority of A.L.L. cases. It was, fun fact, the first cancer mutation ever identified, since it is visible under a powerful microscope by causing some *very* wonky stuff with the 22nd chromosome - they first saw it in 1959.
Previously, leukemias had proven treatable in many cases with the advent of bone marrow transplants: leukemia is bone marrow cancer, so we use intense radiation and chemotherapy to fry your bone marrow into oblivion and then replace it with someone else's who is similar enough to you that you can get by.
But, it didn't really help people with Philadelphia chromosome, which for some reason was much more resistant to treatment. Even a couple of decades ago, other kinds of A.L.L. were seeing 5 year survival rates around 50-60% while diagnosis of Philadelphia chromosome was a death knell; survival rates hovered below 10%. Treatment could usually induce a remission but it almost always came back stronger within weeks or months.
And then, in the early 2000s, a miracle; the development of the first mutation-targeted medication for a cancer, in a weird twist of fate for the first kind of cancer whose mutation was ever identified. Imatinib is a tyrosine kinase inhibitor - a tyrosine kinase being, to my understanding, a kind of enzyme that signals cellular reproduction and which Philadelphia chromosome causes its cells to make a broken version of which is always on, always signaling for reproduction. TKIs find cells that have the broken enzyme and cause cell death. (And also to some amount of regular blood cells too I'm pretty sure since mine makes me continuously anemic and immunocompromised, but I'm not going to look a gift horse in the mouth here, it hits the mark most of the time)
This finally gave patients with Philadelphia positive A.L.L. some hope, as well as patients with chronic leukemias, a much higher proportion of which have Philadelphia mutation origins. People began living longer, and some even began being cured and remain alive today.
There are now two successor generations of TKIs, generation two is dozens of times more powerful than imatinib and generation three is hundreds of times more powerful. Two years ago when I was diagnosed, my doctor gave me a prognosis of not just a 70% chance of 5 year survival, but a 60% chance of reaching a cure - and unlike Philadelphia-negative A.L.L., without even a bone marrow transplant. Outcomes for ph+ and ph- A.L.L. are now more or less similar and it seems possible that in the future, ph+ leukemia could actually be regarded as a favorable mutation for prognosis.
Also part of my treatment regimen (which included more medications than I can count on my hands) was rituximab, because my leukemia also came with the C.D. marker rituximab is able to treat. I'm not too sure on exactly what a C.D. marker is and it seems like neither are most non-hematologist doctors I've asked, but I'm thankful for it, and it improved my chances considerably.
Anyways, cheers to modern medicine! I literally could not get by without it. My leukemia isn't cured - yet, growth mindset - but it's controlled to an extent that it's undetectable on tests most of the time, and if it ever decides to come back I still have several lines of treatment and more powerful TKIs left available to me, even more than I had when I started out.
Minor point but I wouldn’t waste much time with the “patients are just diagnosed younger and therefore do better” line wrt cancer. In many common tumors the cancers typically found in younger people are biologically and clinically different, and often much worse (faster-progressing with a shorter prognosis), than those typically found in older people.
(Breast and lung especially should really be thought of as multiple disparate diseases at this point and imo it makes this kind of broad conversation difficult to illustrate with them, let alone with “cancer” writ large; much better if feasible to zero in on a specific, coherently characterizable tumor type that fits your specific example needs. But I do get that this is a lot to ask in a thirty-thousand-foot-view essay about Medicine Y/N lol.)
The claim "medicine doesn't work" is obviously false, but is likely a straw man argument.
There are several reasons to believe that a large part of approved therapies today, either don't work at all or don't work as they claim they do.
I am not sure I agree with Robert here, but I see a few gaps in your arguments.
1. Survival rates. These are heavily dependent on detection timing. Imagine a world where we cannot treat cancer at all, but suddenly discover a technology to detect all cancers 6y earlier. Then the correct comparison would be between 5y survival rates pre-discovery and 11y survival rates post discovery. The cancers we observe post-discovery are much less deadly, on average, than the ones we observed pre-discovery. 5y survival rate would get to a very high level, but it is not clear what the benefit would be as we would be still unable to treat any cancers. Thus, your comparisons of survival rates are biased by better and earlier diagnosis.
2. Bayesian chaining. It is not obvious to me that a claim "medicine does not work" applies only to step 4 of your causal chain. Step 4 corresponds to the "medicines/medication do not work" claim. I think Robert is arguing about "medicine", not "medication", so it is not very nice to switch the goal to "medication" as do in your conclusions.
It is possible that there is a breakdown at steps 2 or 3 in our healthcare as a system that renders it useless while some medicines do work when used properly. I think it is fair to call these step 2 and 3 breakdowns "medicine does not work". For example., imagine a world where doctors make diagnoses completely at random irrespective of the underlying conditions. Then step 2 would be badly broken, and we would not observe good outcomes for the group with greater access to doctors. I think it would be fair to describe this situation as "medicine does not work". (And, yes, I am aware that this hypothetical is not compatible with perfect detection hypothetical in 1)
And don't forget preventative public health measures! These are part of medicine too. Better to not have cholera or dysentery or polio or HBP or excessive LDL (name your condition) in the first place, than to seek treatment after these conditions become a problem. Preventative measures are generally a good thing, and have been driven by biomedical research. Medicine is not only treatment for a preexisting condition: it's also about preventing the problem arise in the first place: and often the interventions are cheap pharmaceuticals. Sun block, statins, polio vaccine, clean water, etc. The counterfactuals here are weird: should we just close all our primary treatment centres and not bother putting in tents, repairing compound fractures, or whatever?
>Even Robin admits this is a real effect; he just classifies it as more physics than medicine.
Does he have the same opinion about really obvious surgery? Clearly everything that ever works is physics in the end, even if we can't discern all the intermediate steps in most cases.
I was raised in Christian Science, a faith-healing cult that avoids all medical intervention (this is an excellent comprehensive documentary on the topic: https://www.youtube.com/watch?v=E7RT4wNhiYQ).
My grandmother died slowly and horribly and pointlessly of a skin cancer which would have been trivially easy to treat (it has a 99% survival rate).
I, and every single surviving family member also raised in Christian Science, now have varying degrees of permanent damage from benign medical neglect.
Had Robin Hanson looked into Christian Science, he would not have formed these conclusions.
I'm surprised the argument centers narrowly on survival. Medicine is also good for improving bodily function and alleviating symptoms.
For example, anti-epileptic medications might be life-saving (I don't know the data), but they certainly help people to live more normal lives. Opioids and benzodiazepines have made the dying process for many people less horrific. Trauma surgery means broken bones can be fixed and a once disabling injury now is only a temporary set-back. There's durable symptomatic control of many chronic diseases with medication (e.g., rheumatoid arthritis, inflammatory bowel disease, SLE, COPD).
I think it's an inevitable epistemic challenge that there are things we don't know now that we'll know in fifty years about the interventions we're currently using - either there are unintended negative consequences or we thought some things were working when they really weren't (e.g., cancer drugs approved via the accelerated pathway end up showing no benefit when tested more rigorously). That's just the nature of how science evolves over time. In the face of this epistemic challenge, though, is it better to do nothing or something? I think that's the question best asked on a disease-by-disease, person-by-person basis. Will this intervention help this person right now? I spend a lot of my time in palliative care helping folks to navigate just those kinds of questions.
FYI Hanson & Caplan are not alone, this skeptical position is also the main point of Jacob Stegenga, Medical Nihilism (https://www.amazon.fr/Medical-Nihilism-Jacob-Stegenga/dp/0198747047) : "This book argues that if we consider the ubiquity of small effect sizes in medicine, the extent of misleading evidence in medical research, the thin theoretical basis of many interventions, and the malleability of empirical methods, and if we employ our best inductive framework, then our confidence in medical interventions ought to be low"
I do not agree Medicine is not effective, but I think his point is more about what dominates in terms of increasing lifespan and the sort of cost-benefit analysis. For example, it could be that all the cancer, cardiovascular etc medicine we have only increases lifespan on aggregate by like 2 years (because people who have these diseases are likely to die of some other affliction soon enough.) There was a paper arguing that curing *all* cancer would lead to an increase in lifespan of ~ 3 years. I think it's fair to ask: is it worth spending so much money for such small benefits or could this money be used better? I do not think these questions matter a lot because people will still prefer getting medicine, but it's a fair question and I do think we might overestimate the average add in lifespan from medicine beyond antibiotics and stuff like this.
That being said, to me this is the best argument to invest in ageing research.
Clearly reasoned (absence of evidence is not evidence of absence), but are you tilting at windmills? Hanson's perspective seems preposterous. Can we seriously question that diphtheria vaccine has no effect? that antibiotics for bacterial pneumonia are worthless? that insulin is useless for type 1 diabetics? that antihypertensives have no impact on malignant hypertension?
The other angle to do this is doing it bottom-up. Surely dialysis and kidney transplants work against diabetes? Surely getting your broken arm put in a cast works? Surely antibiotics work? Surely having surgeons treat that stab wound works? Surely having your infected appendix removed works? Surely the Covid vaccine worked? Surely antihistamines work? And so on and so on - once you've established that vast slews of medical interventions *obviously* work, then any odd statistics that claim they don't must surely have methodological errors.
As a MD and a researcher setting up clinical trials, I found Robin Hanson considerations absurd and obsolete. It is like discussing if the earth is flat or at the center of the solar system. I think there are far more relevant thought experiment to discuss such AI doctors, AI as a government to avoid needless wars, robot assisted society, space colonization etc.
The Scott Alexander answer is excellent. Vaccines, antibiotics, antidiabetics, antiarythmics, immunotherapy, robot surgery have demonstrated their clinical efficacy in terms of survival rates, quality of life, autonomy and mobility etc.
Okay, can anybody explain to me why Robin Hanson is taken seriously about anything? From my view, the one thing he's done is write some SF about emulated humans in a computer. But that seems to make him a Respected Big Thinker.
The guy comes out with "yeah I had my broken arm fixed, but are we *really* sure medicine is any good?"
Now, do we have contemporary theories that will eventually turn out to be like the four humours? Probably, but that will be due to the advancement of knowledge like everything else. Next time Hanson breaks his arm, do you think he'll stay at home and do some deep breathing, because medicine is still on the level of Galen, or will he head to the local hospital?
The only sense I can make out of all this is that he enjoys being a feckin' eejit, I mean contrarian, and taking the "if everyone jumped off a cliff, would you do it do? yes, mom, I would!" position. That's fun, but do it consistently and you don't come across as a gadfly, you seem to be a silly person.
What's next - "is food actually necessary? history shows primitive peoples believed you could acquire the characteristics of a creature by consuming its flesh, hence they would eat the heart of a lion to become brave, we know today this is nonsense, but Big Agriculture and Big Dietician Industry has convinced us we need to 'eat' to live, I propose instead that all you need is air".
I would love to see someone make the same case for 1600s medicine or modern education.
I’m glad to see you pushing back on this claim as it may undermine the urgency for much more investment into medical research. I would like to see 10x the amount of medical research being done. And why not? There is a lot of room for improvement when it comes to reducing the suffering from physical and mental health problems.
You can increase medicine use with no effects on questionnaires or measured health outcomes. When I was younger, I was very allergic to tuna and salmon. I went and reintroduced them into my alimentation. It took a lot of time, probably shot me to the top 5% of medicine use for my age and country, but did not change a lot on how I would have responded or scored on most questionnaires. I was very good a avoiding my allergies, so I doubt my adverse event rate or death rate moved any direction. I was also not very stressed about the allergy or the procedure and would have not scored differently on mood or stress. You could build a case that nothing changed and that it was useless with the kind of metrics tracked by those studies. Except from the fact that I can eat tuna and salmon now, and that while it's probably not the most efficient use of medicine, it still works and I am happy with it.
How quickly we forget! Does he even know that the mid-thirties were once called "the heart attack years"? I saw a huge change in heart disease over my forty years in practice, partly due to less smoking, but also partly due to diet and statins. Another example: when I was diagnosed with CLL, I looked up the stats and they were dismal. Fortunately things have changed, with targeted treatments the outlook is vastly better, and I may even be cured by an HSCT.
Thank you for writing this! I know you said no one takes Hanson and Caplan seriously, but I definitely take them seriously, even I do not always agree with them. I work in healthcare, and though intuitively I thought Hanson was wrong, I could not figure out exactly where his mistake was. This left me with a gnawing feeling that I might just be biased and everything I do for my job is pointless and doesn’t help anyone. This article reduces that feeling.
> This might seem like a silly question: in Europe of the seventeenth century, the average lifespan was in the low 30s. Now it’s the low 80s. Isn’t that difference due to medicine? In fact, the consensus is now that historical lifespan gains are better explained by nutrition, sanitation, and wealth.
Wow. Hanson is committing one of the best-known statistical errors here. First off, it doesn't pass the basic "smell test." (Does anyone *really* believe that 300 years ago we had a bunch of elderly people in their 30s running around?!?) I can't help but wonder, does Hanson not realize just how well-known of a mistake this specific claim is? It's practically a textbook example of wrongly grouping together unlike data.
"Historical lifespan gains" are virtually nonexistent. Modern conditions have improved things a little bit at the margin, but certainly not by decades! That "average" is explained almost entirely by throwing a whole bunch of 0s, 1s, and 2s into the data that skew the overall number far too far to the low side, and then we came up with vaccines and other treatments that did away with infant mortality almost entirely.
Once you separate infant mortality from adult mortality, a very different picture emerges: the ancient figure of 70-80 years for a human lifespan (Psalm 90:10) has remained almost completely unchanged for millennia, and we're just barely starting to improve on it today.
I think saying “no” to “does modern medicine work?” encompasses multitudes, the possible answers being (strongest no to weakest, clickbaity no):
1. It’s all fake and pointless, all medicine may as well be sugar pills, surgery’s just a more drastic alternative to acupuncture.
2. A few bits and pieces work (eg antibiotics) but the same’s true of witch doctor potions and Chinese medicine; most of it’s pointless.
3. Quite a few of the pills work; most are overrated and some are rubbish. Doctors themselves are a pointless bureaucratic hurdle to pill-buying who add no value.
4. Modern medicine has some kind of paradigm/worldview which is better than chance but fundamentally wrong (a bit like Cartesian vertices).
5. The average person’s life will not meaningfully be improved by access to healthcare (even if a few individuals’ lives will; cf. https://slatestarcodex.com/2017/09/27/against-individual-iq-worries/).
6. Health insurance achieves nothing, in spite of the pills working.
7. Health insurance is overrated.
Hanson’s position seems to be 2, but his studies only get him to 6 at best. The study you’d need for 2 is a comparison of effect sizes (or a better measure of efficacy) against placebo for a panel of medicines weighted by how commonly they’re prescribed.
It might be too difficult, especially with easily available data, but maybe some extra stats could make a stronger case from a lot of these studies -- if the main problem is a largeish number of marginal results.
(I'm not sure how much further effort it's worth going to on this topic but I think the below is generally interesting and worth trying.)
Karnataka measured over 80 outcomes and there are various others.
Unlike published RCTs maybe we can hope they don't suffer from selection (publication) bias, after all they seem to have published a lot of non-results.
Hopefully they also published anything that looked like the medicine was harmful!
Case 1 (no real effect): We'd expect to see a range of results, with no particular bias towards positive outcomes and some marginal results in both directions.
Case 2 (beneficial effect of medicine): We'd expect to see a range of results but with a positive drift. It might be hard to produce well powered formal test, though there are some simple things one could do:
a. Plot effect estimates with CIs, maybe there's a visually obvious positive bias
b. A sign test: What fraction of findings suggest a positive effect vs. negative.
@Scott: If you have or know where the get the data relatively easily, I'm happy to poke around, make some plots and try some tests. Maybe with some kind of pre-registration of what to try, once the basic parameters of the data are clear?
> the 1970s state of the art was doctors saying “You should try to stop smoking and eat better.”
They have updated since then. Now it's "You should try to stop smoking, eat better, and exercise more."
Can the correct medical treatment work if administered properly? Yes, for many diseases the effect is massively positive.
Is going to the hospital and doing what the doctors tell you to likely to increase your health? This is where Robin has the chance to be correct. He identifies a huge gap between what we think medical intervention can do (massively increase our length and quality of life) and what it actually does in practice (very specific and often marginal improvements in a subset of the ailments humans have). This gap creates a huge mismatch in expectations which is extremely dangerous. Patients are massively biased towards intervention. They get frustrated with the modest gains of real medicine and will seek more and more intervention to try to get the increases in health they imagine medicine should be able to give them. Intervention is dangerous. Doctors are likewise frustrated with the mismatch between their very high social status and their very modest ability to heal. They will go along with or even encourage the unnecessary interventions. Every surgery has risk, every medication has side effects. If people seek intervention far out of proportion to real benefits, then the iatrogenic effects of treatment can cancel our the small positive effects of medicine. In this way, even if medical treatment is real, the total effect of the medical sector can plausibly be negative or within statistical error of zero.
Okay, to steelman the argument: without antibiotics and the advent of techniques such as x-rays, 21st century medicine would be as ineffective as 17th century medicine, where high mortality rates were due in part to deaths from infections that could not be treated either prophylactically, as with inoculations and vaccinations, or concurrent with the advent of the infection.
Lack of effective painkillers and anaesthesia was also contributory, as well as lack of knowledge about dietary requirements and deficiencies, basic hygiene theory, and knowledge of organ function.
Since they did not yet have advanced understanding, they propounded theories to explain illness and disease which we know now to be grossly mistaken. Similarly, in future, there may be theories which we today assume to be correct or at least to have explanatory power, but further advances will reveal to be mistaken.
That being the best I can say for Hanson's argument, even in the 17th century they did have *some* effective treatments. Medicine was based as much on a practical and pragmatic view as on theoretical; you might not know exactly how foxglove or willowbark worked to cure, but you could see that dosages of these were effective:
https://www.rmg.co.uk/stories/topics/health-17th-century
"Were any important discoveries made in the 17th century that helped improve doctors knowledge?
In the 1620s an Englishman named William Harvey, who had studied at the great Italian medical school in Padua, discovered that blood circulates around the body, the heart acting as a pump with valves to control the flow. King Charles I encouraged Harvey’s efforts after seeing his work. King Charles II, who came to the throne in 1660 after the death of Cromwell, was also interested in everything scientific, including medicine.
In 1661, a chemist called Robert Boyle published a book called The Sceptical Chemist, which described how the body takes in something from the air to breathe. Boyle also established that without this important gas, which we now know as oxygen, animals and birds would die. In 1662, Charles II granted a Royal Charter to the Royal Society and this encouraged scientists to attempt new experiments. However, despite such promising developments, many superstitions were still accepted as truths in the 17th century."
> after you’ve already gotten the cancer, so it’s hard to see how nutrition, sanitation, etc could explain this
If a person gets cancer or a heart attack, it seems like an obvious hypothesis that they might improve their nutrition, exercise, etc.
The fact that this hypothesis isn't even considered in this article is a great microcosm of the overconfidence of modern scientists.
And I'm guessing there isn't great data on it, but it's fascinating how biases rear their heads on all sides of an argument, and most people don't see it.
It seems to me that while so many people appeal to science, they use poor quality science to rationalize biases and intuitions, rather than saying, "Hey, the science here sucks. Let's bring everyone together from all sides and design a great study to evaluate." COVID is a perfect example: the world was ready to run RCTs on a massive scale and all we got was tiny, crappy ones and decisions mostly based on non-RCTs.
I believe that the workings of healthy bodies aren't studied nearly enough. Unless I've missed something, the result is treatments usually drugs) aimed at relieving symptoms, but the treatments have serious side effects, and sometimes weird side effects that aren't related to the ailment in any obvious way.
There are billions of people who don't have schizophrenia and don't have dry mouth, either. How does this work?
I realize it could be hard to get financing for what seems like abstract knowledge that isn't closely related to a specific illness or a cure for it, but abstract knowledge can end up being very important.
I have no strong opinion about how much of medicine as now practiced is worth it, though I would be on more than Hansen says and less than people who trust the whole system think.
I'm sleepy, so I might just be being stupid, but this doesn't seem 100% coherent at first glance:
> "In Robin’s model, these extraordinary studies would have to be bias or chance, and totally coincidentally at the same time somehow better nutrition made leukemia patients (but not uterine cancer patients) twice as likely to survive."
If we grant that the study/medicine is somehow bullshit, then surely we're granting that the survival rates you're referencing here are bullshit data? Or maybe I'm misunderstanding the association between this data point and the research in question. Like I said, I'm sleepy.
Fwiw I’ve read a lot of what Hanson wrote about this, and my impression was always that his framing was “medicine doesn’t clearly work in aggregate”, instead of “no individual type of medicine works”. My understanding might be right or I might be implicitly taking a saner-but-wrong interpretation.
In any case, I think there is a very good question here: if we assume medicine really, obviously works, e.g. by using arguments in this article (and tbc I do), then why can’t we support this with population-wide sociological studies? We spend a very significant fraction of our total wealth on medicine, and it should distress us that we can’t (or simply don’t try hard to) find a high-level aggregate effect.
"Medicine never works" is sure catchier than "some medical interventions don't work very well," but it is only the latter than has a chance of being plausibly defended. When I was in my late 30s, I ruptured a disc in my back at L5-S1. This was excruciatingly painful, to the point that even sitting was close to impossible. I had to lie on the waiting room floor prior to surgery. I had a very successful laminectomy, without which I would be walking with a cane and taking opioids every day of my life. If this happens to Hanson, what's his plan? If he really believes medicine never works, then he might as well go to a witch doctor, or "traditional Chinese medicine," or do nothing at all. Get a cane and some oxycodones, I guess.
Without taking an overall position on the debate (yet), a couple of points:
"[This paper] finds that “plan effects on blood pressure” were three times higher for hypertensives for non-hypertensives; that is, unlike statistical flukes (which we would expect to affect everyone equally), the effect was concentrated in the people we would expect doctors to treat."
This seems mistaken to me -- conditional on the assumption that blood pressure is the 1 of 30 that coincidentally had positive effects, high effects on hypertensives specifically is something we should *expect* to find.
"Third, you need the diagnoses to result in more treatment (eg blood pressure medication)."
An additional factor that could be a separate step of the funnel or could be lumped into this third step is medication compliance, which we know is often poor.
Generally when I'm making the argument that medicine doesn't work, I look to country data. Time series life improvements don't really impress me when those same improvements show up everywhere in the world regardless of how much medical spending those countries are doing.
Spending a short amount of time looking up some numbers, the US spends some thirteen thousand dollars per capita on healthcare. Ecuador spends around five hundred dollars per capita. Life expectancy at 15 is 80 in both countries.
Reducing the rate of mortality by 0.06% is implausibly high, at least if I'm understanding it right. Preventing 1 out of ~1,600 people from dying is a plausible result from providing those 1,600 health insurance, but that's not what happened. Only about 1% of people who were sent a letter actually got insurance, so that means that about 1 out of *16* people who got insurance had their life saved in the next two years as a result of getting insurance. I don't believe that in the slightest.
I personally know intelligent, educated people who profess views similar to Robin's (so it's more than just two contrarians). Sometimes this is just a basic failure of reasoning: only sick people go to the doctor, so if you never go to the doctor, you'll never get sick. But often, I think it's some form of mood affiliation or virtue signaling: medicine is artificial, unnatural, "Western"; natural, "Eastern", or "native" solutions must be better.
Robin's own stated explanation for increased life expectancy contradicts his argument, anyway. Sanitation is good because unsanitary conditions make people sick. Improving sanitation is straightforwardly a form of preventative medicine.
> Robin’s argument (that medicine doesn’t work) assumes that the only possible failure is at step 4, and that the failure must be a true failure rather than one of statistical significance. But in fact there are failures at every step (although I kind of have to stretch it for step 3), and the authors of the papers tell us explicitly that these are most likely failures of statistical power.
Is that his argument? I think if doctors randomly assigned diagnoses to patients (i.e. complete failure at step 2), Robin's overall argument would be correct, since he's talking about the medical system as a whole, not just medicine.
I wish Hanson wouldn't insist on making the most insupportable version of the argument. It's a waste of everyone's time. If we were debating the medical and cost effectiveness of specific treatments, that would be great. We should do that. Let's not have the college dorm room stoner version of the debate.
The Technology of Medicine by Lewis Thomas, c 1970
https://archive.is/F4NhI
Isn't healthcare very likely one of the main mechanisms wealth actually leads to better health (especially if we're already pulling out nutrition and sanitation as separate mechanisms)?
Maternal mortality would seem to be another line where one could argue that medicine works. Especially potent since it's one of the cases where medicine was very likely a net negative (in the 1700's and 1800's) and has now moved to be a definite positive.
Also traumatic injury, hasn't treatment of, for example gunshot victims, improved survivability enough that many of the contrarian folks (though perhaps not exactly these contrarian folks) have hypothesized that it messes some with the crime statistics?
AIDS medications? AIDS went from a death sentence to a manageable condition with targeted therapies. See also curing hepatitis-c.
Really medicine is such a broad category that, as others have noted, it seems just crazy to treat it as one thing. There are undoubtedly parts of it that 'work'. Treatment of infections diseases (vaccines, antibiotics, targeted anti-virals) and traumatic injury/event treatment would seem to be exceptionally clear cases. Of course there are parts that don't work or are even negative to health, and in some cases we have a pretty good idea what those are, in others it's less clear. And the claim for health insurance working is more nuanced (particularly for the marginal increase thereof, and the specific implementation may make substantial difference). And many parts scream out of better cost/benefit (and not just monetary, consider over screening and associated unnecessary treatment side-effects that are argued around, for ex. breast and prostate cancer). But just saying it doesn't work is nuts.
Feels that it is somewhat fair to say that the marginal expense side of medicine has lost efficiency. But, that is a banal statement that relies on how effective our groundwork of medical treatments even is today? Stated differently, I would expect that research dollars spent on medical treatments will not go as far today; because they first have to cover known advances that we are building upon.
Taking that to a stronger statement that medical treatments don't work is markedly absurd. Just look through a family tree going back a hundred years. Be sure to ask how many kids aren't included because they died before they were 5. Then pay attention to how many died of sepsis in the past.
"everyone knows that glasses help your vision"
Myopia is increasing not decreasing.
I love Hanson. One of the amazing things about him is that, despite his self-identifying as not very good at social things, he appears to be a natural at some of the Dark Arts. This is a perfect example: by the time you're arguing about specific claims you've already lost. You might as well be arguing with the sportscar salesman about what kind of shirt you'll look coolest in driving this convertible. If medicine were as great as some think it, nobody would have to get down into the weeds to discuss at what ages people are getting cancer.
I think Robin would probably agree that a drug works. Not all of them but sure, they exist.
The real question is what is the outcome of all medical interventions as typically practiced by the system? If there’s a common drug that helps leukemia and a common surgery that kills more cvd patients then it helps, how do things actually net out? How could you possibly know?
Probably medical system helps but the big takeaway for me is do we really know? Put this in the category of other questions like Caplans do we know education helps kids?
I come away from this thinking A) at least some medicines and condition-screening "works", and B) providing free health care is a bad deal for taxpayers when the recipients are under 45. The rational health policy this would lead to is one where Medicaid expansion is limited to older people, and to the extent certain types of coverage are mandated for insurance carriers it would be heavily focused on things like annual screenings for cancer and hypertension.
While some of this is reflected in PPACA coverage minimums, it also forces insurance companies to provide care that the majority of "customers" will never use, notably substance abuse counseling, pharmaceutical interventions for addiction, expensive patented pre-exposure drugs for unmarried/promiscuous gay males to avoid HIV, and a range of birth control products. And as a result, we have deductibles that aren't far off from the $5K ones RAND considered "very bad" in the 70s, back when insurance was mostly seen as a way to cover accidents and emergencies rather than an overall care plan.
It also continues to baffle me that the people who seemed the most politically active in advocating for various "Medicare 4 All" style programs in the past decade were the younger Millenials in their 20s and 30s, statistically the people for whom increased access to care does demonstrably nothing and who are decades away from any of the mandatory screenings that have benefits. It leaves me to wonder whether these people are particularly neurotic and overuse health care access, particularly sickly, or whether their immediate concern was for older family members. The people most likely to agitate for student loan relief are people who owe a lot of student loans, but the people most likely to agitate for health care access are the people who don't need it?
I've always understood Hanson's considered claim as "we have a huge overemphasis on medicine" not that "modern medicine doesn't work." That's why he suggested cutting medical spending by half, not entirely. This piece doesn't rebut that claim.
I get the sense that Robin Hanson is trolling people for the lolz.
When I was ten years old, I developed abdominal pain. My mom took me to the doctor, who palpated my abdomen and announced that my appendix was about to burst and I needed surgery immediately.
The post-op recovery sucked really badly, especially for a child (I couldn't laugh, because it hurt so much; I had to eat gross pureed carrot soup and wasn't allowed any fried foods or chocolate for months; I hate carrot soup to this day). But the surgery SAVED MY LIFE. Yes, yes, n = 1 anecdote blah blah, but this particular anecdote is MY LIFE. I'd like to see how Europeans in the 1600s, with their leeches and four humors, would have taken care of my appendix.
Robin Hanson can take his contrarianism and shove it where the sun don't shine.
It seems like the dodge of attributing longer lifespans to “nutrition, sanitation, and wealth” is dumb? Wealth is the dumbest since it manifests as better nutrition and better sanitation (and access to medicine, but we can ignore that for now), so it’s redundant. But even nutrition and sanitation are sus since….aren’t those kind of medicine? Like I guess one could say medicine is only drugs, but is it? Why have nutrition and sanitation gotten better if not for “medical” research?
To me the explanation is obvious. The way insurance works, or is intended to work, is that people rarely need healthcare but would struggle to afford it when they are the rare person who needs it. If a particular condition affects 1/1,000 people and costs $1,000 dollars to fix, then insurance charges everyone $1 and pays for the people who need it. At least in theory and discounting administrative costs, etc.
There are some problems with this simplistic model. The chain of events mentioned in the article is the source of most of it. You can't get people treatment if you don't know they have the disease, which you can't know if they aren't getting to the doctor, which they can't do if they can't afford it/don't have insurance. So instead of 1/1,000 of the population getting treatment, you have a much much higher percentage getting checked for the condition, some proportion getting a more significant follow-up check to confirm, and then some final number getting treated. This final number is likely higher than 1/1,000 because it includes people who are mistakenly diagnosed or who fall below some threshold where treatment is necessary but still get the treatment anyway (which the patient will often demand and medical malpractice makes difficult to say no to).
So the costs that should be a certain figure are significantly higher, and most of the difference is not helping actual patient outcomes. You're still only benefitting 1/1,000 of the population, while spending resources on most of the population.
If he's taking the strong stance that all medicine is bunk, then I strongly disagree. If he's taking the weaker stance that a significant portion, likely an overwhelming majority, of our expense on healthcare is wasted/ineffective, I do agree with that. In fact, I think that explains why we get such small results over the population despite knowing that medical interventions *have* to be working in many cases.
Roughly speaking, I would guess that the first 1% of GDP put into healthcare is likely miraculous in results, the next few percent are very effective, the next few percent are actuarily measurable, the next few are a wash in a cost/benefit sense, and the last few are actually making things worse. Averaged out over the whole cost, we end up being unable to link spending with outcomes in a measurable way.
I think his claims totally make sense if you think about the almost 20% of GDP spent compared to how difficult it is to measure positive outcomes. The problem is not that medicine is worthless/ineffective, but that we spend a majority of our healthcare money on things that even in theory can't help people feel better (diagnostics, real estate, administration) to find the small minority of cases that we can help with relatively cheap medical care (relative to the total cost).
The reason Robin interprets null results as negative results is that studies do not fall from the sky (except natural experiments). The RAND study was designed to detect a certain level of effect. Its failure to detect that effect is a statistically significant rejection of the hypothesis of effectiveness held by the designers. This is a dangerous line of thought, but also having a prior on a non-quantitative hypothesis like "medicine is real" is also a dangerous line of thought. The zero point isn't magic. Medicine has obvious health costs. If marginal medicine is designed to have a 10:1 health benefit to health cost and then it is discovered to only have 1/10th the benefit it was believed to, it is quite easy for it to be net negative.
Lots of problems with this analysis on both sides. Imagine a trial where you give out free aspirin to people and track how their headaches improve. The biggest finding is that people didn't have to go out and buy their own aspirin as much. Okay, so this study wasn't about whether people got aspirin (giving it away for free isn't the same as preventing people from getting it) so much as the financial benefit of having to pay for something or not.
I'm mostly NOT sympathetic to Hanson's point, but the decrease in depression and reduced financial burden might be correlated with "didn't have to pay for health insurance" in the Oregon study. Let's take a charitable middle ground argument and say that at least 15% of health care spending obviously works (especially a lot of interventional stuff people in the comments have mentioned; I'd also add cancer therapies with good overall survival benefits, like imatinib, but probably not most modern medicines).
On the other side, let's say at least 50% of health care spending is just ineffective cope for "your lifestyle choices made you unhealthy and we're going to pretend to fix all that". We'll set aside the other 35%, for now, as 'uncertain benefit'. If we cut health care spending from 20% of GDP to 10% of GDP, would people be better off?
(Would people be better off if we shifted 10% of GDP to helping people get fit instead? Probably not. Increased nutrition advice seems to be inversely correlated to people getting thin and fit. Either it doesn't work or it harms people. We need good evidence on this one before we roll it out widely.)
The assessments of whether health care make you better off or not are terrible. That increase in HTN meds? First, I would assume the most severe cases start off on the meds, so increasing HTN medication availability should yield diminishing returns. But what were those returns? Lower blood pressure. Really?! Who cares whether their BP is 140/90 vs. 137/87? Patients care about whether they get headaches, or SOB, or more importantly whether they're going to die of a heart attack. It's death and pain that people get hung up on.
Doc: Where does it hurt?
Patient: In the HbA1c.
Nobody does this. People don't care about their hemoglobin's post-translational modifications, the salinity of their urine, or how much triglycerides are in their blood. They care about living long, better lives. So the real question is: do people live better and/or longer with the intervention. Most of these studies don't measure that over the long term, and aren't designed to. Therefore we can't conclude one way or the other about efficacy.
The correct study is to look at either lifespan or overall survival over long time horizons. There's a reason nobody is doing this. That study is impractical and by the time you get the results it's been a half century and medicine has changed. There's no exception that says "bad data can be substituted for good data because it's impractical to do it right". The exception is, "if we can't do it right, then we just don't know."
I'd be interested in his analysis of addiction treatment. One argument would be that after a brief medical detox, treatment does nothing to increase the odds of sobriety. The other argument would be that people who want to remain sober need multiple interventions across many domains such as economic, housing, emotional, social, and legal.
I'm stopping near the beginning to comment. My spouse is a highly cited, known medical researcher of integrity, and he has been saying the exact. same. things. for years. Our educated friends dismiss him out of hand in spite of hard data and extreme expertise. He thinks statins made a difference in quality of life late in life, and they meet the criteria for validated research. He likes my hip replacement; surgery matters. Antibiotics in moderation matter.
But in general, much medicine and especially socialized medicine is an elaborate grift at the top, supported by well-meaning people who do not question simplistic, conventional narrative.
There are a some things in medicine that work. Vaccines. Anti-hypertensives. Statins. ACE inhibitors. Pap smears. Post-MI care. There are a million studies showing disease-specific as well as all-cause mortality benefits to those interventions. (And some studies that challenge those, but the preponderance of evidence is in favor of mortality reduction.) But there are a million devices, medicines, tests, and woowoo for which we either have no evidence or evidence of no, and which we let the medical system sell with impunity. Population health improvements have been striking. Covid barely made a dent in what we have been able to achieve over the past 200-300 years. Some of that is clearly nutrition, sanitation, etc. And some medical technologies clearly help spin the wheels underneath those mortality reductions but beyond the obvious suspects, we do not know what else is helping. Just some ballpark numbers: out of about 1 million devices, medicines, and tests, maybe 1000 are clearly effective. That's 0.1% with clear benefits. That's enough to account for the majority of the ~2-3mo improvement in life expectancy yearly. But it's also not enough for someone to claim with pretty good certainly that >99% of the stuff in medicine is probably nothing (and I include placebo in nothing).
I largely agree with your conclusions. People are far too quick to assume that more expensive insurance leads to better treatment. Medical insurance is actually very disconnected from health outcomes, as you show in your funnel. That is why expanding health insurance yields tepid results.
I think the problem is not that modern medicine does not work. It is largely about diminishing returns per marginal dollar invested, and the fact that there are other factors that are far more closely liked to health outcomes.
Genetics and healthy habits do far more for long-term health than medical treatments and insurance.
What he actually proves is that intelligence does not make you smarter.
It will save a lot of effort for readers of this blog to recall that Scott is not pedantic and therefore the phrase "claims that medicine does not work" is an extremely valid rhetorical description of the fact that Hanson is arguing that the marginal value of health care is literally zero - even dismissing the Obamacare study with significant statistical power
Furthermore if you do find the 'marginal health spending is not relatively valuable' argument to be persuasive and wish to defend it you should probably support the concept that the argument should be specifically made in a more defensible and persuasive manner!
> Likewise for weight loss - the 1970s were in the unfortunate interregnum between the fall of methamphetamine and the rise of Ozempic.
This is such a banger line lmao.
I too see the 20th(and a solid chunk of the 21st) century as a corpulent interregnum between famines and Ozempic
I own the book and flipped to the relevant chapter, and the section about modern medicine opens with this sentence, "Medicine today is different in one crucial regard: it's often very effective."
I suspect he will just say that he doesn't believe that medicine doesn't work and that this is a strawman.
I don't have the book in PDF, so I'm going to great pains to write these quotes, but here's more.
"The big historical improvements in medical technology don't tell us much about the value of the marginal medicine we consume in developed countries. Remember we're not asking whether some medicine is better than no medicine, but whether spending $7000 in a year is better for our health than spending $5000. It's perfectly consistent to believe that modern medicine performs miracles and that we frequently overtreat ourselves."
Do most people think that higher life expectancies are due to medicine treatments? I don't think most 70 year olds have had a life saving medical procedure during their lives.
A lot of money in medicine is spent doing stuff "just in case" (like unnecessary tests, MRI's, check ups) as well as making the experience more pleasant for the patient. You could design a maximally cheap health care system from the top down where patients can't choose their physician, can't see a specialist without fulfilling specific guidelines, don't get access to any examinations that aren't evidence based, where there are long waiting periods for everything that isn't urgent, -- and this would save a lot of money probably without statistical detriment to outcomes. Patients would hate it though. (This is what public health care is like in some countries.) A lot of people with enough money or private insurance would still spend a lot of money to get more "luxurious" care (like going to a specialist right away or getting an MRI even when there is no strict medical need.) The point is that only part of the money in medicine goes to medical outcomes, per se.
I mean, if European countries that spend about 10% of their GDP on health care get about the same outcomes as the United States that spends nearly 20% (and has a higher GDP per capita), it is pretty clear that theoretically the United States could save a lot on healthcare. You don't need to try to argue that "modern medicine doesn't work any better than blood letting".
Even the oldest known surgery, trephination, worked, as fossils show the trephined lived lives after the procedure.
Don't statins pretty neatly bust Hanson's claim?
Heart disease is a top killer. The NNT_5 for the absolute lowest risk group on statins is 400.
NNT_5 is too short even, because statin benefits compound over decades.
Statins are also cheap and well tolerated
Given higher risk groups have a lower NNT, and people will be on them for decades, aren't we likely saving millions of lives?
Trephination is still done occasionally today. When a person gets a concussion, the brain swells inside the cranium. Trephination relives the pressure and minimizes brain damage. No doubt there were many, many, head traumas in early human evolution, so it seems reasonable that the procedure was thought out and its usefulness deduced. I forgot to mention that trephination is known in early human history all over the world, so it's unlikely it was local religious myth of some sort. References available in my Fundamentals of Cognitive Science (Routledge).
The quote starting with "The rate of mortality among previously uninsured 45-64 year-olds" seems to have had some copy/paste errors, there are missing letters and extra spaces inserted.
I'm dying from recurrent / metastatic squamous cell carcinoma of the head and neck (R / M HNSCC_frustrating), so the sections about cancer in particular stand out to me. Regardless of the current state-of-the-art for cancer treatment, personalized and mRNA vaccines are likely on the verge of revolutionizing cancer treatment.
Take the HNSCC that's killing me: I got a partial glossectomy in Oct. 2022. Mine had some high-risk features, but I was assured that, with radiation therapy, it wouldn't recur In retrospect, I obviously should've done chemo and radiation, but at the time I was pleased to not need chemo, and I foolishly didn't look deeply into the data on recurrence, which is common for HNSCC, and I didn't seek second opinions.
Docs are reluctant to impose systemic chemo because of the side effects. But Transgene has a personalized vaccine that is supposed to prevent HNSCC recurrence: https://www.nec.com/en/press/202304/global_20230418_01.html: "In the head and neck cancer trial to date, all patients treated with TG4050 have remained disease-free, despite unfavorable systemic immunity and tumor micro-environment before treatment," And most of these personalized vaccines have essentially no side effects.
Moderna's mRNA-4157 platform also looks good: https://jakeseliger.com/2024/04/12/moderna-mrna-4157-v90-news-for-head-and-neck-cancer-patients-like-me/, not only in R / M HNSCC, but in melanoma and lung, too. Right now mRNA-4157 is only being tested in the recurrent / metastatic setting, as far as I know, but the logical time to use it is probably when initial surgeries are done: cut the cancer, sequence it, and then vaccine against it to prevent recurrence.
Right now, from a society-wide perspective, the healthcare I've been getting probably fails the cost-benefit test (apart from the fact that the data I'm generating for clinical trials helps move the state-of-the-art forward). My quality of life is low, and while treatment has been extending my life, it almost certainly won't lead to remission. And even if a clinical-trial drug somehow leads to complete remission, I'll never be able to sleep or speak normally again (https://jakeseliger.com/2023/08/27/on-being-ready-to-die-and-yet-also-now-being-able-to-swallow-slurries-including-ice-cream/). A few months ago my brother casually referred to me being disabled, and I was momentarily confused: Who was he talking about? But he was in fact right: I'm disabled and unlikely to ever be able to think or work in the way I did before losing my tongue.
But that should change! Part of the reason I'm so frustrated by the FDA is that mRNA-4157 and TG4050 should already be available for HNSCC. Instead, they're stuck in trial hell, while HNSCC patients like me suffer recurrences and then die.
I remember the 2008 Oregon experiment. I assume that the researchers had to throw something together very quickly if in order to take advantage of the opportunity to perform a controlled study with random assignment. That's why they ended up with self assessment questionaires that the researchers acknowledged might not reflect actual improvements in health, and measures of clinical results where the improvements were too small to be statistically significant
After the study was released, David Brooks claimed on the PBS New Hour that it showed that people didn't benefit from Medicaid--but I note that Brooks never announced that he was giving up <em>his</em> health insurance.
US health spending as a percentage of GDP has been just over 17% every single year since 2008 except for two specific years: 2020 and 2021. Those two years obviously were impacted by COVID on both sides of the ratio, and for 2022 the percentage settled right back down to 17.3%.
My point here is not about whether 17.3 compared to 19.7 (the 2020 peak pct) invalidates Hanson's argument. Rather it is that his cherry-picking of an obvious outlier year, and then rounding it up rhetorically to represent the norm ("we spend 20% of GDP on medicine"), is enough to me to conclude that he's not presenting any actual serious argument he's just internetting.
Scott concludes: "I think if Robin wants to do something with these insurance study results, he should follow other economists, including the study authors, and argue about whether the marginal unit of insurance is cost-effective"
...fair enough, but you seem to forget what the product insurance (all types of insurance) provides/is meant to provide.
The product is not better health, or a promise of better health.
The product is peace of mind. The product is the knowledge that if you need health care, in particular expensive care, it will not bring economic ruin to yourself or your family.
This is the reason health insurance is usually popular, even in population segments that are seldom in need of health care.
If health insurance should turn out also to improve health outcomes, that is great. But if so it is an added bonus. It is not the point of insurance, public as well as private.
Maybe he doesn't consider insulin a medication because it's technically a hormone. Maybe he excludes *all* life-saving and life-extending treatments and medical procedures.
If he did this, there may be a compelling economic argument. But that's not the argument he's making.
Modern RCT trials are allowed to assume side effects are exactly zero if the P value of side effects is <0.05 between the control and treatment arms. Because of the base rates of other diseases, this can lead to treatments passing RCT that are neutral or even negative on net.
I am broadly with Hanson on this one.
Most treatments have very high "number to treat", being in a hospital is dangerous because of chance for MRSA, surgical errors, etc. drugs studies get to ignore side effects unless they are really obvious, often they can get away with garbage proxy metrics as has been discussed by Scott often this blog.
RFK Jr (yes, I take him AND Bryan Caplan seriously) makes an interesting point in his little-noted most-recent book, The Wuhan Cover-Up, that vaccine research may kill as many people through lab leaks as they do, or might, save through effects of the resultant vaccines. Think COVID-19, though there are MANY others, obviously of smaller scale. The lab leaks are, indeed, VERY common, and potent, too.
Then, of course, there are capers like Anthony Fauci's with AZT, and so on.
I wholeheartedly endorse beating up (metaphorically) of all contrarians all the time.
There is a massive difference between "medicine doesn't help" and "lowering the cost of medicine doesn't lead to large jumps in health outcomes."
For my everyday life, my use of medicine is bottlenecked by inclination and time (as I have no pressing medical issues). For the majority of emergencies, I am similarly lucky enough to be able to afford healthcare. It's only that when the medical issues is very rare or the benefits slim that my insurance is likely not to cover me.
>It would be easy to round Hanson’s position off to something weaker, like “extra health care isn’t valuable on the margin”. This is how most people interpret the studies he cites.
>I’ve spent fifteen years not responding to this argument, because I worry it would be harsh and annoying to use my platform to beat up on one contrarian who nobody else listens to. But I recently learned Bryan Caplan also takes this seriously. Beating up on two contrarians who nobody else listens to is a great use of a platform!
Does Caplan really think medicine is generally useless, or does he merely espouse the mostly uncontroversial position that "extra health care isn’t valuable on the margin?"
All he writes is that:
>I’m puzzled. The idea that the quality of health care has improved is already almost universally accepted. Economists who argue that healthcare expenditures are wasteful are self-consciously challenging this standard view, highlighting surprisingly strong evidence that marginal medicine fails to improve health. See Robin Hanson’s “Cut Medicine in Half” for details.
Not only does he reference that marginal benefit of medicine, his actual point is almost the opposite; that the conventional wisdom is that the quality of health care has improved. In that context, although he makes passing reference to Hanson, it isn't even clear that he espouses the weaker fairly uncontroversial version of the arguement. If, e.g. he thought there were a 25% chance that (the weak version of) Hanson's argument were correct, that could still qualify as "surprisingly strong evidence," given that he notes that the position challenges the standard view.
Hanson's point is about whether medicine works. Rather than add in the complexities of insurance coverage (RAND, Oregon and Karnataka) to refute his point, you could probably have just said "Look at the UK, Canada and other countries where healthcare is (effectively) free - or at least where the population doesn't worry about massive medical debt (contra the US studies) and knows how the, possibly new to them, system works (contra the Indian study) and lo and behold, health outcomes have indeed gone up MASSIVELY in the last century."
I mean, the article would have been shorter and more boring, but my first thought when I read Hanson was "Using the US isn't helping your case."
This isn't original to me, but I think it's worth flagging: five-year survival rates after diagnosis can be massively improved just by diagnosing earlier, even if you don't do anything to meaningfully improve cancer survival, assuming increasing lethality over time. I'm not saying that we've done nothing to make cancer safer, but I'd prefer a chart that looked at X years of survival after a Stage 3 diagnosis (or a more closely matched comparison).
I'm sure someone else has invoked the famous Orwell quote elsewhere in this comment section; but in case they haven't - "some ideas are so absurd that only an intellectual could believe them".
There's a trap that contrarians often fall into, of seeing the justification of outlandish propositions as some sort of cognitive challenge. Something like, "proving black is white is really hard, but I reckon I could pull it off." At some point such people are so caught up in their own cleverness that they lose track of whether what they're defending is actually true or not.
I’m generally very sympathetic to your side of the argument. However, if I was Hanson’s lawyer, exhibit A might be the opioid epidemic. It can arguably be traced back to the over prescription of opioids, and has had a massive negative impact on health in the US, an impact that wouldn’t show up in a health insurance RCT, because the epidemic eventually spread throughout society. Maybe medicine has or could create other issues like this (more resilient diseases?)? As counsel I would also discuss the direct evidence for medical care causing deaths through accidents, infections, etc. I don’t think these things would net out in Hanson’s favor, but would be interested to see you address them.
Bryan Caplan tweeting: If I know robinhanson, he's writing a response to acx/slatestarcodex
as we speak. https://twitter.com/bryan_caplan/status/1783253216399880380?cn=ZmxleGlibGVfcmVjcw%3D%3D&refsrc=email
I appreciate you going into this topic. Hanson's position has always seemed both solid and extremely challenging to fit into any consistent model of the world.
(It would also be great to have a more general post on "How much evidence should we require before giving credence to '[entire area of study] just doesn't work' theories?")
I am biased in Scott's favor. But I was a bit disappointed that his argument seemed to proceed as if Hanson claims that there have been no improvements in medical outcomes over time. The question is not how much things have improved, but what caused it, which I think is harder to say. Maybe some of those studies address this, and it seemed too obvious for Scott to hammer on it (or I skimmed over his hammering). I have been surprised by things before, so I wish I thought this really settled the issue. If it did, I missed it.
I find it odd that economists - the pseudoscience to beat the bands - produces people who not just attack other more scientific fields (although medicine isn’t a very hard science) and prove their lack of statistical ability while doing so.
Does the contrarian ever explain why he had surgery to fix a broken arm instead of going medieval on it?
Being myself a physician, I tend to underplay medicine's success, but even I see that the main thing here is perspective. There's a bias to see anything that transitions from "cutting edge high tech" to "low tech" as obvious and commonplace, as "low hanging fruit" in retrospect. as something that was always there.
In the logarithmic graph of medical success, nothing will ever beat the 4 main things that brought us to where we are today - Sanitation, Obstetrics, Vaccination and Antibiotics - all are now so low tech so as to seem divorced from medicine, but they did originate as medicine technology. We can add exercise and nutrition as additional ones today.
The rest of medicine suffers from availability bias - if you don't know anyone close to you that has had a heart attack, stroke, cancer, physical trauma such as a car crash, appendicitis, need for an organ transplant, one of the plethora of autoimmune diseases, etc., it is quite reasonable to think the whole field is useless. But I think that's the hallmark of a functioning system, conceptually - it's invisible to you until you need it, and then it provides.
I'll join the chorus of voices saying that both Hanson's opinion and Scott's refutation are underspecified in exactly what phenomenon they are evaluating.
But consider some specific cases: not just obvious things like antibiotics for infection, but on one hand, take the random person whose debilitating ailment is alleviated by a very specific medicine. If you know older people you can probably name one. Then consider the other random person who takes a daily regime of a dozen pills prescribed by a variety of doctors, who can barely manage to figure out which pills they have already taken or not, and whose liver or kidneys or brain are slowly being intoxicated by the combination. My quick take is that #1 is beneficial, and #2 is harmful ("iatrogenic" is the word nowadays). But you can't easily have one without (some of) the other. By hypothesis #1's medicine is highly specific, so if you under-invest in medical care, patient #1 is out of luck and might as well kill themselves. But #2 is basically a question of *quality*; what they need is a good doctor to review their entire regime and rationalize it. So the balance points towards "we want better medicine" rather than less of it.
The other thing, if you take the broadest possible view, is that the entire modern-industrial-capitalist environment is making us sick and then turning around and selling us dangerous remedies for it. I can sympathize with this view. Barbara Ehrenreich (look her up!) explains it at length in her book Natural Causes. But at this kind of level it's hard to go from global diagnosis to practical applications. And it's not like pre-industrial people were known for their longevity and great health anyway.
It seems like something went wrong copying this quote or it's just shown incorrectly in my browser (firefox 124.01) .
> Using treatment group assignment as an instrument for coverage, we estimate that the average per-month effect of the coverage induced by the intervention on two-year mortality was approximately -0.17 percentage points. We caution, however, that the magnitude of the mortality eect is imprecisely estimated; our con dence interval is consistent with both moderate and large eects of coverage on mortality. At the same time, our estimated con dence interval is su ciently precise to rule out per-month eects smaller in magnitude than -0.03 percentage points, including the estimate from the OLS regression of mortality on coverage across individuals.
Seems like the fs got replaced everywhere except at the first effect where there also is an additional weird character.
Copying it directly from the paper "con dence" I get the same result. However, the text is rendered "correctly" in the pdf.
> I’ve spent fifteen years not responding to this argument, because I worry it would be harsh and annoying to use my platform to beat up on one contrarian who nobody else listens to. But I recently learned Bryan Caplan also takes this seriously. Beating up on two contrarians who nobody else listens to is a great use of a platform!
I'm out of the loop here. Is this intended to be playful ribbing? Because it sounds a little mean. I'm aware Hanson is a proto-LessWronger but not really aware of who Caplan is.
One problem with the funnel (especially for marginal care) is that at some of the steps, there's some probability that the effect will be bad. I go to the doctor more often when the doctor can't help me, and one of those times I catch the flu from one of the other patients. The attempts at diagnosis do me more harm than good--I have an allergic reaction to the contrast agent for the CT scan, for example. I get put on an additional drug for a marginal thing (the doctor gives me antibiotics for my cold to placate me) and have an allergic reaction to them.
I have no idea how big this effect is, but I'm sure it exists. And we should expect to see it have its biggest impact when we're going to the doctor for marginal stuff. If I suddenly go numb on one side of my body, my worries about medical error making me worse off are *way* smaller than my worries that I'm having a stroke; if I have an annoying cold and go to the doctor for it, the probability he's going to do me harm is higher than the very low probability he's going to do me any good.
I think you are arguing a straw man version of Robin Hanson's view. He does not claim that all medicine is ineffective. His claim is that the helpful treatments are offset in the aggregate by harmful interventions. I refer to the latter as "Hansonian medicine." His explanation for observations that greater access to health insurance does not improve health is that it increases access to harmful interventions as well as to beneficial ones.
Interesting post! I think the claim that all medicine doesn't work is too strong but you should check out the book "Medical Nihilism" for a nice Bayesian argument against efficacy of most drugs. Tldr: there are like 30 drugs doing all the work while the rest are free riding on that reputation.
Killer read. Thanks Astral : )
My response: https://www.overcomingbias.com/p/response-to-scott-alexander-on-medical
I'm a HUGE fan of Robin Hanson, and for that reason I'm all the more thankful for Scott tackling the parts of his writing where Robin seems obviously egregiously contrarian to the point of being just wrong.
Thank you, Scott! Please engage Robin in the future, as well.
I cracked laughing out loud at the following:
"I’ll follow Robin’s lead in dismissing the entire medical literature - every RCT of every medication or treatment ever published - because it might have “huge biases,” and try to rely on other sources."
Maybe one of the reasons for low effectivity of medicine are side effects from prescription drugs!?
https://brownstone.org/articles/prescription-drugs-are-the-leading-cause-of-death/
Why are people getting cancer younger now?
Just wondering.
Hanson and Caplan are not undermining the stereotype that libertarians would say the Earth was flat if that's what it took to reduce taxes.
My mom eats like a saint and takes her blood pressure medication and she still struggles. No, she's not secretly cheating or something, I know how she eats. I have heard a lot of stories like this. Maybe the proven isn't medicine but blood pressure medicine?
I think the antihypertensive point furthers Robin's argument to a certain degree. Last time I checked, while lowering blood pressure is associated with lower all-cause mortality, the effect is pretty minimal. Don't get me wrong, there is a pretty significant *relative* effect, but the *absolute* reduction is minimal.
I think this distinction is what Robin's getting at most of the time. He's never really arguing medicine doesn't work, it's more like the cost of it outweighs the utility of it most of the time because our great treatments are not so great at extending lifespan in absolute terms.
Please do Ioannidis in the same fashion. Basic idea, science has issues, but science works.