Since you're probably going to see this comment - I have a random question: I'm using the sun lamp for 30 minutes when I wake up, at 9 AM. If I instead wake up at 9:30 or 10 or 11, should I do it then instead, or skip those days? Thanks!
Speaking as a sufferer from SAD who has used light boxes in the pas (although rarely these days, as I have found it more effective to adjust my lifestyle to have frequent walks in the sunshine whenever possible):
I would not worry about the exact timing too much. It's not really that finicky and precise of a thing. Remember it's just artificial sunshine; so if you wouldn't be afraid to go to the park on a sunny day at 11 am, you definitely shouldn't be scared to use the sun lamp at 11 am!
In my personal experience light therapy is moderately effective to fight depression, when taken at any time of day. The main reason to do it in the morning is to prevent it from keeping you awake too late at night, because it resets your circadian rhythm.
Think of light therapy like a dose of caffeine---if you take it in the late afternoon or evening, you might end up staying up a lot later than you intend, or sleeping poorly (though unlike caffeine, using the light box early enough in the day will probably actively help you to sleep better in the night). But you probably wouldn't angst about taking your cup of coffee at the exact same hour every morning, unless it is really important for you to have a totally predictable schedule. (This seems not the case for you, if you regularly allow yourself to sleep in.)
You can even use your sun lamp in the late evening if you are having a really severe depressive funk and care more about ending it than going to bed at a reasonable hour (but be aware that getting good sleep is also helpful for treating depression). Or more reasonably, you could turn it on for an hour at 4 pm if you are in a place where the sun sets that early in the wintertime, and you wish it hadn't done that. (Maybe in that case it would be better to use it more as ambient light cheer rather than spending a long time staring into it.)
The risks of light therapy are pretty low, so there is no harm in experimenting and finding out what works for you. So try using it at 11 am, and if it doesn't cause any noticable problems for you, then you're good!
I find it unlikely that explaining to someone that the internet can be used to do something illegal is itself illegal. Then again, I don't know what weird laws the US might have...
I suspect the details are variable based on a whole host of factors know one is quite sure of? After all, there are (or at least have been) states where cannabis is illegal, including medicinally, doctors can't prescribe it, but they are allowed to recommend it.
Many of our weird laws are there to give authorities enough rope to hang anyone they deem hang-worthy. Actually trying to follow all of them would make you feel like Jamie Lannister complaining about how many vows knights have to swear.
This is a side issue, but there really needs to be a single word or short phrase for "we are pretty sure this medical practice works, the existing controlled trials are not good enough, we will never be allowed to conduct the trial because no one will let us have control group". Examples:
- Diphtheria antitoxin in the early 1900s. There is a section in the book The Microbe Hunters about how the initial trial had lower than usual mortality but no control group, but now (written in 1926) we'd never refuse to give a child antitoxin so we can never do a controlled trial. So like, we're pretty sure this works but we'll never know.
- Surgical masks to block spread of disease. See Scott's posts back in March 2020.
- Apparently, Omegaven rather than soy-based competitor.
I suspect there's many others, even excluding obvious ones like "bandage wounds instead of letting people bleed to death".
If everyone is convinced that leeches work, then clearly it would be unethical to not use leeches, so we can't ever gather the evidence we need to stop using leeches, unless we get lucky with some kind of natural experiment where the hospital's shipment of leeches got lost in a hurricane. I think from a utilitarian perspective it is almost always worth it to have a control group even if you have a strong prior for the treatment being better than the control. Especially if the consequence of not having a control group is that it will continue to be illegal for almost anyone to use the treatment. The control group is no worse off than they would be if they weren't in the study, so go ahead and have a soybean oil based control group.
The difference is that leeches, in your hypothesis, are an ineffective treatment that is already deployed across the entire patient population. The picture is very different if the drug or procedure we want to test is something newly-introduced: in cases with massive and obvious effects, the *past* can serve as the "control group" showing what happens with patients who don't get the new drug.
I recall there was some doctor in New York with a case series of HCQ on covid patients that looked like a massive, obvious improvement to the SoC, but then people did RCTs and it didn't work at all. So at least sometimes the biases of a non-randomized sample can make it look like a big effect when it's not.
You could easily do RCTs of parachutes on inanimate objects or squirrels (without killing the squirrels!)
Hardly anything in medicine is as simple as the physics of terminal velocity, though. You probably can't work from first principles and get as strong of a prior for any medical treatment as you'd have for a parachute. Personally I wouldn't trust any drug without an RCT. _maybe_ I'd trust mechanical ventilation without an RCT, but that's iffy (what about damage to the lungs, circumventing all the homeostatic mechanisms for CO2 levels that control the PH of the blood, and doctors getting overly paranoid about SpO2 of 93% when it's not a real problem until SpO2 drops below 85%? How do we know what is the correct SpO2 level to make mechanical ventilation worth the risks, anyway?)
I was assuming you'd drop them off something high e.g. a crane, like they do when testing parachutes on Mythbusters, since it's less expensive and more controlled than a drop from an airplane. But yeah maybe the airplane itself introduces confounding factors that need to be considered for a real world efficacy trial for parachutes...
Also, you're talking about safety trials, but in the Omegaven case what was (and apparently still is) missing is an efficacy trial. In other words, Omegaven has been given to loads of babies safely in Europe and there isn't really any doubt about its safety. The problem we have is that we don't know exactly how effective it is because no one is willing to run a study in which they know some of the babies are going to be denied a parachute.
Several of these are "compassionate use" so they have tiny numbers (e.g. 2 infants) and there's not really enough data to say 'a resounding yes for Omegaven'. It seems to be helping, but even so some adverse effects are recorded.
Good news, I just performed an RCT of parachutes on squirrels, and both groups were fine. I even did a second RCT on mice and both groups were fine. Now it's time to move onto human trials. We're recruiting volunteers, would you be willing to sign up?
:)
Even in the medical context, lots of drugs have worked on mice but not in humans or vice-versa. At some point we either have to do an RCT on humans or we have to assume that they'll work on humans when they worked on the non-human trial.
We've been monitoring this biomarker called "terminal velocity" across species, and it looks like as the size of the species goes up, the terminal velocity also goes up. If we extrapolate that line out to humans, they'll probably go splat. But our parachutes were able to reduce terminal velocity by 95% in all the animal trials, so let's go ahead and try one on a human :)
Squirrels have a terminal velocity of 40km/h, but you can't scale that up to humans. By that nonsensical logic, a blue whale would have a terminal velocity of 700km/h! This is clearly nonsense. Let me write a longform article about how your field is filled with fools and needs to divest itself of the "terminal velocity fallacy" in order to be taken seriously.
I suffer of a chronic disease without any real cure. So quite interested in research. I stop reading studies when it turns out it's a mice study (some are amusing though, like Viagra curing good knows what...)
Viagra was originally heart medication, with erections as a side-effect. It wouldn't surprise me that increasing blood flow generally could be beneficial for all sorts of conditions, particularly anything that's more like wound healing than fighting off an infection.
I agree. People might object that this would enable snake oil salesmen, but people are already allowed to sell unlimited snake oil in the US, without even proving safety, so long as:
1. they call it a "dietary supplement"
2. they use weasel words to describe its effects
3. the substance can be found somewhere in nature
4. the substance has never been investigated as a potential drug.
None of those criteria have anything to do with consumer protection. It would make sense to replace all of them with "just prove it's safe".
N-Acetyl-Cysteine just got taken off the market after half a century of safe use because someone at the FDA noticed that someone was investigating it as a potential drug in 1963.
That's not exactly how it works. There is a large number of "general recognized as safe" (GRAS) ingredients for which no FDA approval is required, roughly because they have been consumed for a long time and nobody has yet noticed anything evil. (Whether the ingredients are "found in nature" is neither here nor there -- plenty of genuine drugs, e.g. taxol or penicillin, are natural products -- it's a question of whether loads of people have actually been eating it for decades.)
That's why you can add salt or gelatin or vitamin C to a supplement for which you claim health benefits but *not* have to submit a drug application. People were eating those ingredients for decades and even centuries before the FDA was formed, and nothing bad was noticed, so it makes little sense to attempt to regulate them now. Although obviously that does *not* guarantee they're harmless, cf. cigarette smoke, in use since the 1600s. Nevertheless, the FDA's ambit by statute doesn't include GRAS ingredients, so caveat emptor and away you go.
You sure do have to include weasel words in your advertisements if you want to claim specific health benefits. You can make generic claims like "good for you" or "healthy!" all you want, but if you say something specific like "lowers your blood pressure" or "cures cancer" then you do need to add a specific disclaimer that says you haven't asked the FDA to evaluate your claims, because people might naively otherwise assume you wouldn't make such claims unless some massive RCT study had been done and the results certified by the FDA. (The fact that this annoys the neutraceutical and alt-medicine Laetrile aficionados is to my mind evidence that they indeed would like to free-ride on the reputation of real pharmaceuticals for rigorous testing. If they thought FDA approval meant nothing to their customers, or wouldn't be otherwise assumed, they wouldn't give a damn about adding the disclaimer.) But having to add disclaimers is separate from being able to use GRAS ingredients.
NAC got taken off the market because by statute any ingredient which first appears in a consumer product as an FDA-approved drug cannot ipso facto be GRAS. In this case, NAC was approved as a drug in the 1960s, before it was used as a supplement, so it can't be GRAS. That the FDA hasn't rigorously enforced any restriction of it until recently doesn't change its legal status, there's no such thing as a "legal easement" where if you get away with a violation of the law long enough, or it's small enough, it becomes technically legal.
Someone could argue NAC should go back on the GRAS list, and I don't doubt many someones have or will, but the way to do that is do the right (expensive) studies and offer persuasive factual evidence, not make an emotional appeal to intuition: "everybody's been doing this for years!" -- yeah, well, people smoked for centuries before someone figured out it gives you lung cancer and heart disease, which is kind of bad.
This article is fantastic. I also note that it uses the "parachute" terminology already - in particular, to argue that many practices which are claimed to be parachutes are not even close to being parachutes.
Honest question - in Science! am I allowed to use natural experiments as a source of data?
EG in 99.9999% (a few famous people lived) all known cases since the dawn of heavier than air flight of falling from great heights and an aircraft in flight *without* a parachute has resulted in death. In our work with parachutes, 99.9999% of our our research subject survived a fall when protected by a parachute avoided death.
Can I use those two collections of data to reason from like a control and not control group? To avoid exactly the silly argument described here that there is no RCT on parachutes?
Presumably one can and should, in those situations, assemble the evidence one can in the absence of RCTs such as retrospective comparative studies, which still could be persuasive if the effect is large enough (as it would be if ifs a moral imperitave to not do RCTs)
I attended a research talk about mitigating the harm of sepsis by activating an enzyme (Heme Oxygenase 1) naturally occurring in the body. The presenter basically claimed it worked miracles, and i found it convincing. Three years later it has not entered common use.
Something similar happened to me, where the FDA did go faster than their normal process, but still nearly killed me in the process. Dan Elton, of the US Transhumanist Party, wrote an article about it:
See also, a whole lot of people who died of COVID in November-December 2020, while the FDA was evaluating clinical trial data (albeit at "warp speed").
Worse than that - scheduling the meeting and getting the right people to look over the reports usually takes much longer. The COVID vaccines got pushed to the front of the line, just like they did for every intermediate approval. Three weeks really is warp speed for the FDA.
What's doubly depressing is we're repeating the exact same mistake with the third-dose booster shots against the Delta variant, and vaccinations for younger children.
Reading your story, it's not "The FDA almost killed me", it's "the lawyers almost killed me" because the pharma company went in for legal wrangling with the FDA over the blackbox warnings which few people ever read:
"Even though the FDA had granted Enyvio Priority Review Status, for the next five months the FDA kept John (and undoubtedly many others) suffering as they debated the wording for the warning label. Since the approval of earlier monoclonal antibody drugs such as Remicade and Humira, scientists learned that in the long term some patients can develop allergic reactions to treatment. The FDA wanted to put a warning about the possibility of a severe allergic reaction on the label. The problem was that the drug had been designed specifically to avoid the allergic reaction rejection problems that occurred with Remicade and Humira, drugs which did not contain such a warning. A warning about allergic reactions would make Entyvio look less safe, something that the company rightly took objection to."
Of course the company objected to it, but you can equally say there "if only the company had agreed at the offset, it wouldn't have taken five months of lawyers exchanging letters to get this done".
And if the FDA had let it go on what the company wanted, no pharma company ever wants "possible side-effects" put on warning labels. I have a dog in this fight as my doctor recommended me a medication, gave me the obligatory warning of possible side-effects, reassured me that such was very unlikely then wrote me a prescription.
I read all the warnings on the box, was highly alarmed by the "this is rare but IF it happens you are in deep doo-doo" warning, looked it up online, yes indeed it was a real, rare, but deep doo-doo side effect. "Well, it's rare, I have little to no chance of getting it" says I; I take my first dose of the medication - and I get the starting warning symptoms.
"Okay, not taking any more of this", I say, I tell my doctor why, she tries to talk me round, I refuse, and I get put on a different medication.
And I do firmly believe that, had it not been for the FDA or similar body sticking it out to put the "it's very, very rare BUT" warning on the label I would have squashed my reservations about the initial symptoms because the doctor says it's okay, continued to take the medication, and ended up in deep doo-doo indeed.
While I see your point, that legal wrangling didn't need to happen at the end of the process - as the story points out, the trials and approval process lasts for *years*. A huge part of the extreme slowness of the FDA is that many things that could easily be done in parallel are only done sequentially, and I bet every email between the wrangling lawyers took a full week to get sent, when realistically it could have all been hashed out in a day or two if everyone involved was locked in a conference room.
^ Yeah. I don't think I mentioned this point in my blog post, but should have. The hashing out could have been done long ago.
Deiseach's comment is well taken. I agree 100% -- the more risk information available to the consumer the better. What they should have done is gone back and modified the labels for the older drugs to update them with the new warning info. It seems it was the contrast between the lack of warning on the old drug and the warning on the new drug that the company took objection to more so than the warning itself, although they had a case against that too. Apparently it's hard to update labels, though (?).
I've read varying accounts online of whether the FDA uses rolling reviews but generally speaking they do not. So they don't start crunching numbers on the Phase III trial data until after the application is submitted. I suspect the factory inspections don't occur until after stuff is approved, also, although I don't know.
As another example of how things can be done in parallel, at some point after the initial phase III trial enrollment(s) they allow have doctors to start prescribing it. That is the idea of adaptive licensing, essentially which I summarize here :
That's a very reasonable point in your case, but I don't think it's a reasonable criticism in my case. If other drugs in the class had this risk, but didn't have the warning, and a new drug is SPECIFICALLY designed to avoid this risk, it makes no sense at all to put the warning label on the old drug.
I see a role for the FDA, for all my bitterness I wouldn't necessarily burn it to the ground. But in this fight the pharma company was entirely in the right to insist on no warning label. "If they had stopped protesting something indefensible they could have helped you sooner" is true... but doesn't shift the blame.
What is the actual number of babies that died due to this? Definitely hundreds? (I saw a claim on Twitter it was more like “ten or twenty” than “hundreds or thousands”)
I also question that number. Not because any single life isn't precious, but because it speaks to the profitability of the treatment *and* about the difficulty in finding enough cases for good studies.
Any word on the number requiring liver transplants or otherwise suffering permanent damage? it could be that the larger number is the total number of babies affected rather than the number that died.
A really good question. No system is perfect and if the worst example of this kind of failure mode we can come up with is tens of people dying where we have a treatment that we can’t get over the hurdle of proof of efficacy, then we actually might conclude that we have a pretty good system.
So why were they so sure that "it isn't the lipids"? Because it seems a bit less certainty on that could have helped a lot. Was that a reasonable position? Would the establishment still have the same kind of position in the same kind of situation or has something changed since then?
I would guess it's because biochemically speaking biologically-derived fatty acids are pretty generic, it's just a question of how many carbons you have in the tails, and where exactly the double bonds lie. They're all metabolized in approximately the same way, so far as we know. I think for a long time people just assumed they were essentially all interchangeable, kind of like whether your gasoline contains heptane or octane doesn't matter that much.
The realization that this was not so -- e.g. that unnatural "trans" fatty acids were strangely dangerous, or that it actually makes a difference whether your first double bond is 6 or 3 carbons away from the end (the difference between omega-6 and omega-3 fats) -- seems only recently to have become part of the standard wisdom. I'm not sure anyone has any good mechanistic grasp on *why* this is true, also. The only obvious lesson here is the usual one: biochemistry is absurdly complex and not infrequently counter-intuitive.
Yes, although in this particular case (the infant parental nutrition), the assertion is that it *was* a matter of life and death. That is surprising to me even now -- I would never have suspected that soybean v. fish oil could mean the difference between lethal liver disease and health. I do not doubt empirical observation, but from a theoretical biochemical viewpoint that is seriously weird.
Perhaps it has something to do with neonate liver function, which differs in some important way from adult (or even child) liver function, in which case it's even less surprising that people were surprised by it, didn't expect it. The body of research on how 6-week-old livers work, and how that might differ from how 6-year-old or 60-year-old livers work, is probably amazingly scant.
> The body of research on how 6-week-old livers work, and how that might differ from how 6-year-old or 60-year-old livers work, is probably amazingly scant.
If the differences aren't known, it just raises the question again of why the overconfidence that it wasn't the lipids? Seems like extraordinary hubris to just assume neonatal liver function has no differences. You'd think we'd have learned better than to make assumptions without empirical data by now.
I don't think it's a question of hubris, it's a question of priorities. People are going to be very desperate to try to save a dying newborn, but they have to focus, because you can't try everything all at once, there just isn't time, or the resources. So they have to make some hard choices -- heck, this happens in medicine *all the time,* oncologists and surgeons and their patients make these choices every day -- do we try this or that first? Because once we make this choice, we're committed, there's no do over...
So they maybe say, gee, based on what we know, we think X or Y is more likely than the lipid content of the food. In this case, they turn out to be wrong. But that, too, happens all the time in medicine. Unless you were there, and know what X and Y actually were, and what the reason was for thinking they were more likely to be the issue, it's very hard to second-guess this and be confident they were idiots, as opposed to people who didn't have all the facts we have trying their best.
Even if I agree with everything you just said, the claim wasn't "we think lipids is less likely than X", it was quoted as "we know it's NOT the lipids". These are very different claims.
And even if they turn out to be *correct* that it's not the lipids, that doesn't rule some other compound present in fish oil that makes the difference, so ruling out the fish oil because of the lipids is also jumping to conclusions.
Reading the account in Scott's post they seem to have done tests with soya oil and without soya oil in such formulas, and similar problems happened in both, so they said "well it can't be the lipids since the same adverse results happen even without the soya".
If, for instance, the babies on non-soya formulations had done better, then they would have accepted "okay, the problem is with the soya oil". But of course, it was more complicated than that and not as simple as "replace the soya with fish oil".
I read a little more on this, and the suspicion is that it's a high fraction of polyunsaturated fatty acids, which actually starts to make sense, since each unsaturation is a hiccup in the normal process of beta oxidation of fats, and requires some extra hokey pokey step, and maybe baby livers don't have the latter ability fully developed. But this is just a wild guess.
it's unclear whether the "without soya oil" formulations had much oil at all - it's possible they were testing whether presence of Omega 6 was harmful and missing that the problem was absence of Omega 3; or, more likely, something more complex like the ratios
In the "relative balance" kind of problems, this is definitely true. In the rarer "actual full-blown deficiency, your body is not getting enough to meet requirements" kind of problems, it is not - deficiency of essential fatty acids (a.k.a. "protein poisoning" a.k.a. "rabbit starvation") is the only known form of malnutrition that can kill faster than simply eating nothing.
Curious about PNALD in Europe. This story is a little different than a new discovery of an alternate use for an older drug or something where there isn't already a body of evidence that it works. If Omegaven was already a standard of treatment there, was there a correspondingly lower incidence of PNALD? Might that have been valid data to use along the way?
Yea, for a complete outsider, this seemed weird to me aswell. I had the impression that science is an international endeavour - so wouldn’t studies on outcomes in Europe be valid? The USA babies could be a huge control group for European ones.
Or are confounding factors between continents too large (eg different quality hospitals)? Or is the number of affected babies still too low? Or do the European babies get other problems more often? Something else?
What is missing from the story in this regard? (Perhaps something obvious for someone with knowledge in medical sciences?)
Yes, I get that; but as I understood it the American regulators were apprehensive due to a lack of (good and large enough) studies. Given that different formulas were approved in the different continents - shouldn’t it be easy to write a study comparing the results (and use that to convince regulators)?
Since saying “babies in Europe in this situation survive 4x as often as in America” in a study seems so obvious, I am sure that there is something missing (in my understanding at least - but maybe also in the story).
Not medically related, and nearly forty years out of date, but when I was doing milk powder testing in a creamery co-op lab, it was for export and had to comply with US regulations, and those were more stringent than the at-the-time acceptable ones in Ireland/EU.
So certification and standards did/do differ from one country to the next, and simply saying "it's fine in European country X" doesn't mean it is legally acceptable in the US.
Just to be clear; I never intended to make the argument that regulations/certifications/standards are the same/similar, nor that “it’s fine there” is/should be enough. Moreso I wonder whether the liver disease (or other issues) were actually obviously gone where the alternative treatment were used.
Yes — I was suggesting the same. Not that the European regulations could apply in the US, but that the European market vs. the American market might effectively function as a giant study with a control for children who did and didn't receive Omegaven. It would still need to pass American compliance requirements but might provide a lot more data than the few US trials.
To put it much stronger: shouldn’t there have been a large and obvious difference in PNALD incidence? And wouldn’t that fact alone be evidence much stronger than that of any of the experiments?
I'm about as cowboyish as it is possible to be and still be employable, so of course I thrived at a tech startup and hated working at a giant tech company. The increase in size of companies due to M&A or economies of scale or natural monopolies in tech is probably driving the culture away from cowboyishness.
In the least cowboyish culture I can think of, Japan, for a long time there were giant conglomerates dominating every industry, all headquartered within a few hundred miles of the God-Emperor's palace. In the wild west, there were tons of small businesses far from the reach of centralized authority.
Isn't everywhere in Japan within a few hundred miles of everywhere else, in particular the God-Emperor's palace? I guess you mean "the greater Tokyo area" in which case it makes sense.
It was a derogatory term in Tombstone, AZ, and much of the Old West almost from the start. "Cowboy" basically meant "robber", with a side order of if they were the first settlers in the area and there was *nothing* to steal, they'd set loose some cattle and sit back until they could steal everything the cows had built.
Western novels, movies, etc, rehabilitated the term in roughly the way that "The Dukes of Hazzard" rehabilitated the Battle Flag of the Army of Northern Virginia. And we need *something* that symbolizes the general ideal of disrespecting established authority while still behaving honorably and engaging in productive enterprises on one's own terms. If there's cause for one or two of those terms to go away because of a preexisting and dishnorable usage, that's one thing - if they *keep* getting deprecated then it's going to look like someone is demanding submission to their authority and that all productive enterprises be done on their terms or not at all.
It sounds like your qualm is with how American healthcare law is written, not with how the laws are administered by the FDA.
As an example, who are you more mad at: judges who upheld disparate sentencing guidelines created during the war on drugs, or the legislators who wrote the laws?
My own take: the legislators are the most to blame. The trial judges are most likely often/usually blameless, but appellate judges arguably are not. Disparate sentencing guidelines arguably infringe on the eighth and fourteenth amendments, as well as some states' own constitutions.
Plot twist: some of the legislators who supported those harsher penalties for crack were black congressmen who saw the devastation crack caused in their communities in the 80s. So long as the demand curve slopes downward, harsher penalties = less crack. I don't agree with the drug war, but it's not such an easy mistake that I'd hold it against them.
And I honestly don't know if I could have done any better in their position, with the information I would have had at the time. Nevertheless, they're responsible for the consequences of their actions, including the unintended ones - to some degree that's the price you agree to when you choose to be in charge. Being wrong isn't something we should condemn anyone for, provided it's an honest mistake and people try to learn from them. Sadly this seems to be an unpopular position.
More importantly, all subsequent legislators are responsible for not having fixed the problems caused by previous laws passed.
> Disparate sentencing guidelines arguably infringe on the eighth and fourteenth amendments
Why? As far as I understand, the 14th Amendment forbids unequal treatment based on who you are, not what you do (such as what drug you sell).
What I don't understand about this debate: If powder cocaine and crack have mostly the same effects, but sentencing is much harsher for crack, then why doesn't everyone who sells/uses crack switch to powder cocaine? And are we sure that whatever difference makes people use crack doesn't also justify the harsher penalties (based on the assumption that drug prohibition is justified at all, which I don't think it is)?
crack is stronger and thus probably more dangerous than powder cocaine. It would make sense to have harsher penalties for dealing more dangerous substances.
IANAL, but my understanding is that if you can demonstrate racist intent on the part of lawmakers, then you may be able to claim the disparate impact is in fact a form of unequal treatment based on who you are. Hard to prove, though, just like it's hard to get any results arguing about unequal enforcement of laws against different groups of people.
Disparate impact is part of some anti-discrimination laws such as the Civil Rights Act of 1964. It's less clear to what extent it applies to constitutionality under the 14th Amendment. It looks like it may apply if discriminatory intent is proved, but it's hard to prove.
Thank you for this story of heroes and a system that can be so cruelly imperfect.
I'm glad you plan to write more on this topic in the future, because it's so important, and your analysis of it has been incredibly enlightening. Do you plan to write about the approval process in other countries as examples of ways the FDA can be improved? Or as cautionary tales?
Wait, so the only evidence for the effectiveness of fish oil in treating PNALD is a poorly powered study and a few miracle cure anecdotes? Miracle cure anecdotes are a dime a dozen. You can easily find thousands of them for everything from homeopathy to touching a statue of the Virgin Mary. That doesn't mean homeopathy obviously works or should be approved by the FDA.
"We were confident we were onto something and began to discuss our findings with others at BCH. The response was not what we expected. We were told “Everyone knows it's not the lipids because liver disease happened with them and without them.” Some individuals would even pull me aside to ask “Are the results real?”"
Usually when one scientist is convinced about the miraculous properties of their discovery while everyone else is skeptical, it's everyone else who's right, not the one scientist. I have no opinion on who's right in this case, but I wouldn't be surprised if fish oil turns out to not be very effective at all in treating PNALD.
This is why bleeding and cupping were standard of care for centuries. Hey, we gave it to Patient X and he got better! But Patient Y died! Yeah, well he had [other factor]/[was already at death's door]/[a physician who didn't do it right].
The reason we enforce the unnatural and cumbersome process of double-blind RCT is *because* historically speaking we know it is incredibly easy for human beings to detect "signal" in "noise" and rationalize it with some elegant theory.
> No, it really isn't. Pick a quack therapy that you're sure doesn't work, and Google "(quack therapy name) + testimonials"
No, what you're doing is fishing for an outcome in a sea of noisy data (like p-hacking). What the doctors did was an introduce a specific intervention to a cohort and observing an undeniably large signal. These aren't remotely the same.
A lot of undeniably large signals are also p hacking or are otherwise mistakes. Especially when you don’t know the doctor or how many specific interventions to a cohort they or others tried
P-hacking requires a large data set from which you fish out a signal that suggests an intervention. The exact opposite is happening here: the intervention is made and the signal is immediately seen. I frankly don't know why people can't see how different these scenarios are.
There are of course the possibility of confounders, but this also shows the weakness of the insistence on RCT for everything in evidence-based medicine. Some studies simply can't be done with controls which means some important life-saving interventions will never be approved. It's pure nonsense, and evidence-based medicine advocates need to take the stick out of their nethers until they figure out a better way to handle these scenarios.
There are a lot of people making interventions, and a lot of interventions being made, and a lot of ways for one to make an intervention and conclude it worked when it didn’t. As a result, and for many other reasons, many errant discoveries are made. The discoverers tend to believe exactly as you said there. Not all discoveries are like this, but one should look for more proof than just “person said it work”, even if it seems clear
I periodically suffer from a disease that doctors say is uncurable. But even time I get it, I spin around three times and say "abra cadabra Ave Marias!", and within weeks, my symptoms are gone! The effect size is huge. My friends can tell I'm better within seconds, without even talking to me.
What uncurable disease do I sometimes get? The common cold.
By the way, where did you or Alsadius read that the babies all got better? Scott's post only said that Charlie--aka a single person--made a miraculous recovery. It also said "Someone else in Hong Kong tried to do a randomized controlled trial, but ran into the same problem - halfway through, the parents figured out what was going on, demanded their babies be put in the Omegaven group, and nobody had the heart to say no to them."
...but it says nothing about the outcomes for those babies.
> By the way, where did you or Alsadius read that the babies all got better?
I did the obvious thing: googled "pnald fish oil" and looked at the resulting studies. To be clear, I wasn't claiming that the patients Scott was talking about all survived, merely that the study I found said that none of the 215 babies studied from 2004-2015 given fish oil had died:
"Premkumar et al. (62) reported outcomes in 57 infants treated for PNALD with parenteral fish oil between 2007 and 2011. All infants were under the age of 6 mo at initiation of fish oil and most were born preterm (median gestational age of 28 wk). Cholestasis resolved with fish oil therapy in 82.5% of infants. Ten infants died during the study period, though none of the deaths were due to complications of fish oil therapy or liver failure. Nine of the 10 infants who died did not demonstrate resolution of cholestasis: 3 had end-stage liver disease prior to treatment with fish oil and 4 died after redirection of care due to poor prognosis from multiple comorbidities."
> we gave them hydroxychloroquine/remdesivir/(?)ivermectin and they got better! Obviously it works!
This is only the most recent and publicized version of something much more routine and more common
History (and present) is littered with doctors convinced that their patients prove their treatments work but they don’t in fact. It’s really weirdly common.
Well if you think you can do better than Cochrane and want to smash your head against a bunch of published trials with different interventions, timelines, structures, metrics, and endpoints, there are lots of published trials, many of which end with something like “we found no difference between soybean oil and mixed oils in liver failure outcomes” and many of which end like “we found significant reduction in liver failure outcomes”. I read a few and have no clue
Time to remember Ignaz Semmelweis, who got convinced about the miraculous properties of washing ones hands between an autopsy and midwifery work. Didn't do him good. Of course, physicians are much more rational today, aren't we?
“this drug is too good so it would be unethical to have a control group”
Around 15 years ago, my wife was taking chemotherapy as part of the regimen to treat her colon cancer. Her oncologist asked if she'd be interested in participating in a trial of a new chemo drug. (I don't recall its name or maker, sorry.) She was interested... on the assumption that she'd be in the non-placebo group.
Not long before the trial actually began, she told me they'd decided to change it so that *everyone* would get the new drug. No one with cancer wants to be in the placebo groups!
Obviously not a double-blind study anymore. I'm not sure how the maker managed that switch or what exactly their new trial design was.
Presumably the control group in such a study would be being given the current standard chemo-therapy; it would be insane for many reasons to give them no treatment at all
> It will be really hard to change the system radically enough to make it possible to do better, but I still think we should try.
Lots of people agree with this, but the political coalitions in the US make this impossible.
If you read Charles Murray’s By the People and Krugman’s Conscience of Liberal (both MIT PhDs), they are actually 100% on the same page about who the players are and how the coalitions arose.
According to Murray, he believes the post 1950 government is illegitimate, and the last time America was still America was the 30s. And that people who think like him took over the Republican party starting with Barry Goldwater, via a few influential publications and right wing think tanks funded by a handful of wealthy families (who he thanks individually, and where he has worked most of his career).
Krugman is exactly on the same page, with the people, institutions, ideology, and timeline.
Krugman’s main counter-point is that the broad middle class that emerged after WWII was not the natural outcome of technology but massive interference in the market, and the natural market outcome is industrial-revolution like inequality. Which I am also convinced is correct for reasons too long to get into here.
An ideal state would combine the polices that maintain a less unequal distribution of income (which is probably needed to maintain political stability in democratic countries), with heavy doses of capitalism and markets. There are certainly people in the center-left that support this. But they are in a coalition with the left, because after the Republican party takeover not enough common ground exists to make policies that keep inequality in check.
So, maybe we need radical change. But with these coalitions, what radical change is possible? Where is the radical change going to come from? Until we can answer those questions adequately, we should be careful what we wish for.
Thanks, I haven't read it, I'll take a look at it. The thesis seems plausible for much of history, although it's hard for me to imagine that policy instruments are useless in modern states that have a much bigger economic roles than in the past.
> In any case, economic inequality seems to me a rather dilatory pre-occupation in itself
Regarding inequality there are three major issues that come to mind:
Is it a bad thing?
Is high inequality the natural state?
Why did it decrease after WWII?
To briefly address the first point, it's obvious that equality is not a good on its own - you could have an equal society with everyone poor. The reasonable concern with inequality is if the consequences interfere with having a society that is largely not poor. I think the better description of the ideal is "broad middle class" - what matters is the average standard of living. There could also be secondary sociological / political concerns that Charles Murray talks about in "Coming Apart"; in addition to the economics people are increasingly living different lifestyles and so can relate to each other less well. But I'll leave that topic aside here.
For if it conflicts with the middle class - the economy is not a zero sum game but there are resources that are scarce, such as human labor. People who are employed by high consumption by high income earners are necessarily not available to work on other things.
To go a bit deeper, my favorite lens is sectoral balances. Unless you print dollars, there is a limited amount of money in the system. A $100 loan is a $100 asset on one side and a $100 debt on the other, which adds up to zero. This creates certain simple constraints in thinking about what is possible that don't require any advanced math.
For example, you can divide dollar balances into foreign / domestic government / domestic private sector. It all has to add up to zero. If you have a trade deficit, that means the foreign sector has a dollar surplus, and the combination of the domestic government / private sector must being going into debt.
Similarly, big tech companies with imperfect competition are accumulating big cash balances. Who goes into debt to sustain this? Can the debt be increased perpetually? What happens when it stops? Another factor is the "paradox of thrift" - people may want to save dollars at the same time, but that's mathematically impossible, it has to add up to zero. One person's savings is an other's income, and by trying to save both fall and everyone produces less.
I'll stop there, Ray Dalio has some good stuff if you want to look further. But I think if you analyze things in this way enough it suggests the laissez-faire approach has some problems and thinking things will just work out remotely well on their own is probably naive.
I'd like to add that if things *don't* work out, there are political implications, and there are no guarantees that laissez-faire or many other political things can be maintained. The choice is not straightforwardly between the welfare state and libertarianism.
I'll skip the second issue (is it the natural state) since it seems like you might agree there already.
For the last question, "Why did it decrease after WWII?", Krugman argues tax rates, labor unions, and cultural norms (for example, the unions might object to executives paying themselves $100M, and there was less reason to do that anyway due to the high tax rates).
It's obviously a complex topic, and this wasn't the answer I wanted to hear (I still have some reservations about unions). But sorting through the arguments (and applying additional economic thought experiments) I've become grudgingly convinced that there's a case for policies that are a bit more left-leaning than I was originally inclined to, and also that maintaining freedom, efficiency, and stability over the long term is a very tricky business.
"Another factor is the "paradox of thrift" - people may want to save dollars at the same time, but that's mathematically impossible, it has to add up to zero. One person's savings is an other's income, and by trying to save both fall and everyone produces less."
(1) If you are putting your money in the bank as savings, then that money goes out in the form of loans so it is indeed being spent by others as a form of income and goes back into circulation in the economy
(2) If you are stuffing your money under the mattress, it is taken out of circulation. It is not being spent as income, true; production is less, true; but one day your heirs will find this windfall and spend it themselves.
On (1), if you put the money in the bank, and someone else takes out a loan, that doesn't describe "people wanting to save dollars at the same time" (at least how I meant it). One person is saving and another person is going into debt, and it still adds up to zero.
But in many circumstances more people may want to save than go into debt (for example, if the future economy looks less rosy), and this is where the problem arises. What happened after the financial crisis was the government went into debt, which enabled the saving by the private sector. There's a tendency for people to feel saving is good and debt is bad because that's true on an individual level, but on the macro level they are the same thing - two sides of the same coin. You can't just have everybody save without printing money.
On (2) this only works on an individual level and not a macro level. For example you can't have everybody save up money for 20 years, then have the whole economy retire.
Money is basically a promise for future goods and services. If that person stuffs their money in their mattress, you don't actually *want* production to be less. The important thing on a macro level about being able to fulfill that promise is the ability to produce *future* goods and services. But producing one less car in 2000 doesn't help you produce a car in 2020. Most goods and all services can't be stockpiled. And drastically cutting back on current production may even harm your capacity for future production.
This is why you see some emphasis on infrastructure - of the available things to spend money on, it's one that has some plausible positive effect on future ability to produce goods and services (for example, by people spending less time in traffic).
To tie it back to inequality a bit: wealthy people save a greater percent of their income. On the other side of that it drives the interest rate down, which increases the price of housing and the amount of debt people go into and that balances out the saving. But now interest rates are about 0, and can you easily go lower than that (and keep the debt going up to balance the saving)? And if people want to save but not go into debt, you don't end up saving but just produce less...
Yes, later in life he endorsed it. I myself think Milton Friedman - style libertarianism with UBI is a great compromise between enabling a middle class and libertarian values, it's a simple plan to address the problem without too many adverse incentives, while maintaining small-government elsewhere. It's an extension of the capitalism-with-safety-nets that most successful countries seem to follow, just much more to the libertarian side.
His party isn't with him on that though, and doesn't look like it's heading in that direction. So I think it's not particularly relevant to the political coalitions as they stand.
He endorsed it in "In Our Hands" in 1997 and "In Our Hands" in 2006, whereas "By the People" was in 2015. Part of the reason he endorses just giving people money is that they would then have the freedom to do whatever they want with it... and his complaint in "By the People" is that regulation is inhibiting such freedom.
My point was about changes to the Republican party starting in the 1960s. Murray's UBI recommendation is something that happened later, and the Republican party hasn't adopted it.
I think what you are talking about, UBI and major deregulation (Milton Friedman was against professional licensing for example), is a compelling idea. There would certainly be advantages and disadvantages, but I think it would likely be better than the current system (although personally I might want to run a small-scale experiment before scaling it up). If the Republican party focused on that instead of, for example, corporate tax cuts, I think they might be more popular overall and it would be more beneficial to the public. Unfortunately I don't see any reason to believe things are heading in that direction.
Uh, I didn't say they were? Previously the nominal rate was high but the effective rate was substantially lower than the nominal rate. And it looks like VAT makes up on average 20% of tax revenue in the OECD, behind income tax and social security contributions (as of 2018).
Look, it's not like I've never seen a Republican policy I liked. Immediately before the rate cut bill, they were attempting comprehensive corporate tax reform (with a border adjustment tax) which I thought was a clever idea that addresses the tax haven issue, without needing any international cooperation.
But even just in terms of pushing libertarian principles, I think it would be better for them to prioritize middle class tax cuts and reducing costly regulations (and they do do *some* of this). Both because that's more in line with academic economic thinking, and from a basic intuitive standpoint that corporate earnings and the incomes of the wealthy were already at record highs, so borrowing a lot of money to increase their after-tax incomes further didn't seem urgent.
As it is the establishment is losing credibility with the base, and they more or less didn't even campaign on the tax bill even though it was the main piece of legislation they passed. And to the degree that the new faction that is replacing them has a coherent ideology, it doesn't seem to be particularly libertarian.
There's a big difference between the published timelines and the real timelines. I only have to deal with the FAA, not the FDA, but it's safe to say that the problems are endemic.
Nobody every got fired at the FDA because a PDUFA was responded to a few years later than the published guideline.
Why is not having a control group considered such a deal breaker? I know it makes it hard to tell if there was a placebo effect, but if an experimental treatment is much more effective than a placebo possibly could be, shouldn't that be enough?
The issue isn't so much about placebos as it is about selection bias. When you do the study, some percent of the patients of the patients will recover according to your success criteria. What do you compare that percentage to in order to determine if the drug is effective?
If you compare to global or nation wide results, there is the potential confounder that not all hospitals are equal. If you compare to results at the same hospital at different times, there is the confounder that medical practice changes over time.
Whoever you compare to is your de-facto control group. If they aren't randomly selected from the same pool of patients, there is a significant selection bias associated with who is participating in the study. This is all before you consider the question of placebo effects.
If the drug is so good that almost everyone who doesn't get it dies, and almost everyone who does get it lives, that may be good enough without the explicit control group. But your evidence will almost certainly be weaker than you expect. Intuitions are misleading in such cases.
I definitely agree, particularly in this case. I just want to separate the scientific question of how to evaluate the strength of our evidence from the moral question of how much evidence is necessary before a drug should be permissible to proscribe. I think the FDA's failure is largely on the latter question rather than the former.
I think I missed something. In Europe they use fish oil based nutrition. So wouldn’t rates of PNALD be much lower in Europe? Is the advance that it has to be pure fish oil?
Retrospective studies are always difficult and rarely definitive, simply because it's extremely difficult to match your experimental and control groups. It's unusual for the required demographic and medical information to be fully present, and to be sufficiently confident that everything *other than* your experimental variable was held constant. Remember this is medicine, and moreover a rare condition, so we are inherently in the realm of very small and very noisy data sets.
Amazing. This ultra obsession with randomized controlled trials is crazy -- complete brain dead. Come on I get it. If you want to pick up very small effects yes you need them, but if your expectation are big results just use some bloody common sense. You will be able to see them if they do exist. The base line is that all babies die. If you save 20/100 you have your answer in front of your eyes.
> If that's the rate, you need to have a sample of hundreds of babies with digestive tract issues to have statistical significance.
Fair, but follow-up studies showed *none* of those 1/100 babies died in the timeframe from 2004 to 2015. I think the signal is pretty clear even without an RCT.
The use of double blind studies of sufficient sample size is critical. But for many kinds of large scale research, the system is breaking. Mostly because the hurdles to get through multiple studies of incrementally larger sizes, and different designs, and different standards of proof. All the while trying to weigh risks and benefits for the control and intervention group.
I'm most familiar with how to evaluate genetic effects and risk factors in large populations, This is maybe a good example of how it works in medical interventions.
There's a better, more analytically principled, and powerful approach (we use this for optimizing game settings and ad placements in my current job). It's to use (usually Bayesian) models of likelihood to probabilistically assign treatment. Like, if there is a high risk, maybe-good new formula, you'd assign it at very low probability. If it has great results (the likelihood it's better goes up) and you start prescribing it more. As the sample size goes up, but the results begin looking worse, you assign it less.
Importantly, you don't need to give it to thousands of people off the bat and risk so many lives. But you can ramp it up quickly as it becomes more obvious that not doing so risks lives.
That does sound like a great solution! It won't work for everything since there have to be people willing to try it though ("here's this drug that might cure the common cold or might kill you"), but it seems like it would've at least been useful here.
The problem is you're assuming in this scenario that the question "is it working?" is easily answered. And that may well be the case in many fields, but it is most certainly not the case in medicine. There are a host of reasons, from the psychological (in both patient *and* physician/researcher) to the statistical and even definitional ("what do we actually precisely mean by 'working?') that make this a major challenge.
One of the most important reasons for doing double-blind prospective studies is to prevent these factors from having too much influence, and in particular preventing the inevitable human psychology effects in situations of enormous emotional weight from screwing up the results. People have *a lot* on the line in these trials: the patients are often betting their lives or health, the physicans are betting their self-respect and peace of mind, the company may easily be betting billions of dollars or their entire corporate existence. If the trial was not blinded, as you suggest, and things were continually evaluated openly, it would be very difficult to prevent these psychological factors from playing a big role.
I mean, after all, this entire post stream grew out of the feeling that the aducanumab approval was driven by too much emotional and/or financial investment in a positive outcome.
Reread my first paragraph. The issue isn't the double-blinding. It's the need to start with a "big enough" sample size, get all the data at once, and do several full rounds of the analysis-different proposal-grant-experiment loop.
By the time you've gone through this a lot, it's possible to lose steam, or fail to get funding. Or see a few null results. And this can all be hobbled by experiments that use very weird baselines, null hypotheses or just engage in p-hacking.
Updating Bayesian priors on the fly, with transparent likelihood models, and risk-benefit correction is a lot more tractable, and effective.
Yeah, but see I'm disagreeing with you. I'm saying *without* blinding, you are basically going to generate noise, and not any useful signal, because of human nature, and any effort to optimize the efficiency of generating noise is ipso facto wasted.
OK, then I'm disagreeing with the opening sentence of your 3rd paragraph, which begins "There's a better, more analytically principled, and powerful approach..." I don't think the approach you describe is better, because I don't think it would work, because blinding/human nature. Sorry for the lack of clarity.
You can still use a blinded design with the approach I described. You SHOULD use a blinded design with the approach I described. Blinding good. Sample size good.
bad: "Mostly because the hurdles to get through multiple studies of incrementally larger sizes, and different designs, and different standards of proof. All the while trying to weigh risks and benefits for the control and intervention group"
Aducananumab would be a good example of how you would do the Bayesian approach, and why an artificial threshold like "FDA Approval" is totally broken.
You can start by offering the drug to some people, and sure, as there are few bad side effects at first (it's not gonna kill you) and it's clearing up some plaques, you'd ramp up the amount on offer.
But over time, you're going to see (again, sure, it's not going to kill you) and maybe okay, you can approve it, but the marginal benefits on long term cognition are basically 0 (it's as good as the control). A simple likelihood model, with a cost factor, would quickly tell you not to prescribe it.
We probably would have started ramping up vaccine distribution 6 months earlier, for the mRNA vaccines, under a similar dynamic testing regime.
You have to have some sympathy for the FDA as they're being simultaneously criticised for not bending the rules because BABIES WERE DYING and for bending the rules because GRANNY HAS ALZHEIMER'S.
Don't respond to heartstrings-tugging: heartless paper-shufflers
Do respond to heartstrings-tugging: brainless jellyfish
The objections are because Aduhelm hasn't any evidence that it does any good for Alzheimer's but for the fish oil there also wasn't any evidence that it did any good (the warning about efficacy are still up on sites about it):
Limitations Of Use
Omegaven is not indicated for the prevention of PNAC. It has not been demonstrated that Omegaven prevents PNAC in parenteral nutrition (PN)-dependent patients [see Clinical Studies].
It has not been demonstrated that the clinical outcomes observed in patients treated with Omegaven are a result of the omega-6:omega-3 fatty acid ratio of the product [see Clinical Studies].
I very much want to hear about what the default-yes and default-no-but-with-smaller-hurdles models look like to you! *refreshes inbox excitedly for those posts *
I feel like it may be worth noting as a small aside here that soybean oil consumption has increased significantly in the US in the last several decades and is put in, well, just about everything I look at here. Meanwhile, fish oil is not only much less common in our standard diet, but so uncommon that it is taken as a dietary supplement by millions so that they get a small amount instead of almost zero.
I think it's likely the case that at least one, if not more, of the following is true:
1) excessive soybean oil (and similar oils, perhaps high in linoleic acid) consumption is very common and very bad for you
2) same as the above, except that it is primary bad for you when used at high temperatures such as for frying
3) the ratio of omega 3 to omega 6 fatty acids in our diet is terrible (with most of us having far too much omega 6 in comparison), partly as a consequence of the large prevalence of soybean oil and large lack of omega 3 oils such as many fish oils, and this is probably very bad for us. This may also be why fish oil supplementation seems to help some groups of people while doing nothing for other groups of people
I'm still only moderately confident in some of these, but they do seem to be relatively convincing propositions the more evidence I find in their favor, and soybean oil is probably not the best oil we could be giving to an infant regardless here, so I kind of frowned when I read that it was the standard, when it is also the first ingredient in a lot of pretty terrible things. Conveniently, it is also extremely cheap, so perhaps I should not be surprised.
It seems like the new recipes for neonatal lipid formula is are mixed and include soy and fish oil. So I’m not sure that it’s a case of “soy oil bad” as much as “soy only is bad”
Also, for the natural health perspective, I think many peoples of the far past lived inland and were quite primitive hunter gathering on lots of natural animal and plant fats and no fish at all.
That said I don’t eat isolated plant oils at all and do lots of fish and animal due to more general suspicions of modern food, but the whole oil composition thing has always seemed like the evidence isn’t strong enough either way to tell.
In various farmed fish mixed soy:fish oil preparations are much better for growth than either soy or fish oil alone, and 100% soy also increased liver fat. I was looking for studies on people, but thanks google!
A combination is probably a decent bet, I intended to convey that point with possibility number 3 in my comment; that it may be the case that soybean oil is not harmful unto itself, but rather that an overdone ratio of its constituents may be.
As usual it's very difficult to have high confidence on these propositions when most of the good studies we have are not in humans (or don't even exist to begin with!) and the data we have in humans is generally terrible and only correlational, but I do think it's a nice area to at least spend some time looking in, agreed.
Yeah, unhappily it's not until *after* people start keeling over dead that anyone is motivated to find out "oh crap, soya oil in high quantities is not good for you".
It's a lot easier to take it as read that soya is safe, because look at all the countries that have been eating lots of soya and soya products for centuries and they're not keeling over dead.
Indeed not, the Japanese eat a metric crapton of soy every year, and they have among the highest life expectancies and lowest rates of heart disease in the world. If memory serves they *do* however have higher rates of liver cancer than we do...
Don't the Japanese also eat lots of fish oil every year, though? My understanding is that Japan is a massive outlier among developed countries in how much fish is commonly eaten.
Yup. God damn it, messing up the experiment that way. We need a God-Emperor who would command each country to differ in its diet from other countries in only *one* particular...
An explanation I've heard is that whole soy foods (edamame, etc) are safe, but processed soy foods aren't. Same goes for things like canola oil, the actual oil isn't so bad but the processing to extract is so harsh that it becomes unhealthy.
If there is a harmful effect from excessive O6, it is likely some longer term (over decades) increase in various types of cancers (via systemic inflammation?), and perhaps subtle mood disorders. But there are few control groups since practically everyone uses "vegetable oil" nowadays.
I've nearly eliminated excess O6 from vegetable/seed oils in my diet, it was pretty hard actually, it is used in a LOT of foods as a fat substitute, and is ubiquitous in restaurant food. Fresh bread is one product where you can easily tell the difference between oiled and non-oiled, oiled is "harder" especially when toasted.
Like basically all health diets, the benefit probably isn’t the O6 but the fact that you stopped eating restaurant foods and the sorts of foods with fat substitutes. That’s my guess for what the real benefit is. Idk
I'm generally skeptical of arguments that ordinary food processing processes are making any chemical changes to food constituents, since these generally require harsher conditions.
Most of the time I suspect any benefit of less processing inheres in avoiding changes in the mix of nutrients, which among other thing scan interact in interesting ways with your own gut microbiome, or possibly with changes in the microbiome that lives on the food itself, e.g. I might hazard a guess that fresh carrots plucked from your own backyard garden and rinsed under the tap are better for you than factory farm carrots power-washed in a giant machine because those from your backyard still have a variety of healthy soil bacteria on them.
But that could just be because I'm biased to think the biology matters more than the chemistry per se, and/or because I have a vague suspicion the influence of our bacterial ecosystem (in our guts) is way larger than we have thought -- which may help explain otherwise baffling aspects of the nutrition/health matrix: it matters not just what we feed *our* cells but what we feed the billions of bacteria in our guts, and how that changes their relative demographics.
Re the hunter-gatherer diet. One traditional way for some coastal indigenous cultures in the US was drying or smoking fish, making fish jerky. Then it lasts a long time and can be traded. Also, inland rivers and lakes used to have many more fish than they do now; I don’t know about relying on fish but accessing fish should not have been difficult even for non-coastal groups.
I have theories about regional human adaptations to high mineral content in water sources. I don’t know of any studies on it though.
I just meant that a decent proportion of primitive peoples didn’t eat any fish, and skimming a few papers seems to confirm that (and many ate some fish and many eat a lot of it... the only real constant in the behavior of ancient/primitive peoples is how much it varies, and humans seem adapted to that)
Digestion is complicated! It makes sense - I was googling just now, apparently there are 3 subtypes of omega 3 with plants (nuts/seeds, algae, a few other things but not soy) being a viable source of at least 2.
It could have come down to something as simple as the USA has a lot of soya crop production while Europe doesn't, so when making up the formulations the German company went for "well we can easily get fish oil".
And if I go by the warnings, it's not uncomplicated story of "fish oil good, soya bad" because Omegaven has adverse side-effects as well:
"Risk Of Death In Preterm Infants Due To Pulmonary Lipid Accumulation
Deaths in preterm infants after infusion of soybean oil-based intravenous lipid emulsions have been reported in medical literature. Autopsy findings in these preterm infants included intravascular lipid accumulation in the lungs. The risk of pulmonary lipid accumulation with Omegaven is unknown.
Preterm and small-for-gestational-age infants have poor clearance of intravenous lipid emulsion and increased free fatty acid plasma levels following lipid emulsion infusion. This risk due to poor lipid clearance should be considered when administering intravenous lipid emulsions.
Monitor patients receiving Omegaven for signs and symptoms of pleural or pericardial effusion."
At the same time, other websites are claiming soya oil is good for you, though they do point out the Omega-6 imbalance:
Fish oils aren't the only source of omega 3 fatty acids, they're just unusually concentrated sources and also provide the most-readily-usable omega-3 fatty acids. There are plant sources, like some seeds and nuts. Even beef, chicken, and eggs would have traditionally been good sources, but the amount of omega-3 fats in them depends on the diets of the livestock, so grain-fed livestock have much less than pastured animals.
"One small weak randomized trial wasn’t enough evidence for anyone else to care. But it was enough evidence that BCH refused to do larger studies, because that would require putting some babies in the control group, and obviously those babies were going to die, and that would be unethical."
Not just drugs. It took a long, long time for the "lumpectomy" approach to treating breast cancer (as opposed to radical mastectomy, taking everything) to become standard of care. Because who the hell wants to be in that experimental group? "You've got a cancer. This has always been treated by removing the entire breast, but we want to try just removing the tumor and a little bit around the margins. It might be just as good as taking the whole breast! We'll look at whether those in the control group live longer than you. Want to sign up?" Er...yikes.
It's a heart-warming story and I am very glad that now there is a proper feeding regime for babies like this.
But the bit about deliberately putting pressure on the pharma company by feeding negative PR to the media? That's how you get Aduhelm and drugs like it approved.
And the praise of cowboys - the reason "cowboy" became a term of opprobrium is because people have experiences of such types. Authoritarian surgeons who think they are 'saving lives' have caused suffering - there's one notorious scandal from the late 90s in Irish medical history:
Wouldn’t a better way forward be to grant medicines approved in reputable places like the EU, UK, Japan or Australia an automatic provisional authorization?
That's been floated many times before – and it IS a good idea. Alex Tabarrok is the person I'm most familiar with who's brought it up 'recently', and repeatedly.
Sadly, it probably just doesn't work from a 'public choice' perspective, e.g. the first bad 'approval via proxy' would probably see it immediately reversed.
His father was a rabbi in Columbus Ohio in the 60s and 70s at my grandmother’s temple! I think Dr Folkman, famous in the Boston area, actually got caught up in some scandals about his research, but I don’t remember how it all got resolved.
Even if you count from the first study in humans to the ultimate clinical trial(s), total time is 9.1 years.
But that isn't because of the FDA per se, it's because biology is really hard, and we don't want to kill people. It isn't trivial to introduce a drug to humans for the first time, as the TGN1412 study tragically showed: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2964774/
Here's an easy way to minimize the risk of initial human trials that would never, ever, ever be possible, even though it clearly makes all kinds of sense: give it to terminally ill patients first, offering significant amount of money to them and/or their heirs. Of course, terminal illness is a huge confounder in the big picture, but it's a great way to at least make sure that the drug won't necessarily set one's lungs on fire.
What makes you think it's easy? It's already difficult to enroll enough very sick people in clinical trials for a therapy that might actually help them, I mean, where the payoff matrix at least includes one square with a positive number. You're going to try to enroll people in clinical trials where *none* of the squares have a positive number? Tricky.
And the immediate problem with offering large cash awards is that your enrollees will quite naturally be disproportionately poor and minority patients, most likely those without health insurance, people on Medicare/Medicaid, et cetera. Let us imagine your proposal passes and we find that these kinds of medical experiments, with the occasional gruesome results, are predominantly being performed on poor and minority patients. Let us imagine the political campaign that immediately follows the sensational news stories...
I mean, basically it's the same category as offering $1 million in cash to the family of any healthy young person who signs up to be a heart transplant donor, in order to solve the shortage of heart donors. You can imagine what the political response would be to such a proposal.
Yeah, I think it’s ridiculous anyone thinks that’s a reasonable objection. Of course people with less are more incentivized. That’s true everywhere and all the time at every level of wealth. It doesn’t mean poor people are cows without moral agency.
Hmm...well, OK, but it's pretty clear the rest of the world violently disagrees with you. You cannot sell yourself in to slavery, or answer Armin Meiwes's ads. If Union Carbide decides to cut corners in its Bhopal plant, but not the plant in Missouri, because the much poorer people in Bhopal (1) don't object as strenuously, on account of the good jobs at the plant, and (2) when a lot of them are killed in the terrible accident they settle for much less than the Missourians would, the world condemns Union Carbide as immoral bastards, rather than bidding up their stock because of their clever business strategy.
> I realize the hurdle is there for a reason. What would a default-yes medical world look like? What would a still-default-no-but-the-hurdle-is-smaller medical world look like? I have thoughts about this which I hope to address in future posts.
Given the history behind the term "snake oil salesman" and Coca(aine)-Cola, my presumption is that there used to be such a world. What were the events that led to the reversal? What lies behind Chesterton's Fence ought to be historical record, at least in part, right?
But it seems that today "low effectiveness plus high safety" is rejected. That category scoops up "not enough proof of effectiveness yet plus high safety" and so it's the efficacy hurdle that might cause the most problems of the type Scott described regarding Omegaven. "High effectiveness and high safety" can take a long time. "High effectiveness and low safety," I am ok with rules around that type of thing, although what type of rules is debatable. But I also think it is ok to put gates in Chesterton's Fence (once one understands why it was built), partly because in this case, the edifice really did accrete, it was not dropped on us in complete form but evolved over the course of a century in response to competing pressures. Those pressures aren't over, so it can keep evolving.
The European regulatory body the EMA functions differently. Here they are compared, "The FDA requires valid scientific evidence that devices are both safe and effective. By contrast, the European CE mark requires only proof of safety and that the device performs in a manner consistent with the manufacturer's intended use." (Retina Today, Sept. 2012, Shapiro & Lafond). So my impression for a while has been that more things get through in Europe. Like Deiseach said elsewhere they might not have specifically rejected soy in Europe, the fish oil formulation might just have been the locally developed thing which then turned out to be more effective than the US one in those situations.
Even the FDA sees that it may need to evolve its effectiveness axis:
""The randomized clinical trial is excellent methodology if you want to understand, on average, whether one treatment is better than another treatment," notes John Bridges, assistant professor at Johns Hopkins School of Public Health, "but if we think about a distribution of outcomes, no single person in the health care system is the average."" (FDA.gov)
> But it seems that today "low effectiveness plus high safety" is rejected.
I wish there was more rejection of that. As long as you have high safety, low efficacy therapies, doctors will gravitate to them due to sheer risk aversion. As a patient, I 'd rather take more risk for higher efficacy (assuming the disease is serious in any way)!
I clicked Scott’s link to the SSC post on the research study he had been doing that was swamped by the IRB. Very interesting. Here is an article that concluded what you were suspecting.
Cerimele, Chwastiak, Dodson & Katon, 2013, “The prevalence of bipolar disorder in primary care patients with depression or other psychiatric complaints: a systematic review.” Psychosomatics, 54(6).
Screener - 20.9-30.8% prevalence. Full diagnostic interview- 3.4-9%. Whee.
Have not fully read this but I guess primary care has the advantage of repeat patient contact and permanent staff.
I'm really straining my memory here, but I think that the screening instruments were designed for a primary care environment but getting used in psych wards. CCD&K proved that they didn't even work for the primary care environment they were designed for. I wanted to show that they didn't work in psych wards either.
This is a really excellent post, and I commend your willingness to dig deeper into the story and add a thorough and thoughtful correction to your original story. It was instructive to read this, not only in terms of learning what actually happened, but in seeing how the story was oversimplified and in some cases distorted over the years and among a wider audience -- which is alas all too common.
Indeed, I think one of the obvious conclusions to be drawn here is that when it seems like some large group of pretty smart people has done something just so incredibly brain-dead that it only takes a moment's thought to realize how they went wrong, and how it could have been done better, that some uncomfortably high percentage of the time that conclusion is flawed -- that there is much more to the story than was at first apparent.
Of course, sometimes people, even very smart and experienced people, do make stupid or corrupt or evil decisions, so we can't rule it out all the time. Still...if I had to make a choice, I'd say the modern era could usefully move its collective needle towards patience and away from quick judgment.
The usefulness of the FDA has been studied a lot, and every study I've read has said it kills a lot more people than it saves, but gets away with it because it kills individuals by banning drugs that will save them, which doesn't seem as heartless according to American biases.
Well, as a dyed-in-the-wool empiricist, I am automatically deeply skeptical of any study that relies on counterfactuals -- evaluations of what the world *would* be like if the world we live in were radically altered in some way. So this kind of evidence is pretty weak by my standards. Doesn't mean I wouldn't be persuaded, if there were enough of it.
Part of the obvious problem, to my mind, is that predicting what the world would be like if the FDA behaved quite differently is very challenging, since it's so different from the world in which we actually live, and more importantly there are always dynamic effects, meaning you can't *just* say "What if the FDA approved more drugs with lower levels of evidence *and nothing else changes?" Other things *always* change, because people respond dynamically to changes.
Id est, if the FDA starts making it easier for drugs to be approved, drug companies will respond to that...in some way. Physicians and patients will respond...in some way. If the approvals are good, people will respond in some way, and if there are more that are bad, people will respond in some way.
Doesn't mean I don't think changes should be contemplated, by my preference would be to do things experimentally and incrementally and see what happens, rather than make any big change, especially one driven by hypotheticals. I think there have already been efforts along these lines, e.g. the "fast-track" approval protocol that got implement in the 80s when people were enraged about how long it was taking AIDS drugs to get approved, the "compassionate use" protocol that allows desperate people to try stuff, the "emergency use authorization" that allows them to signal some kind of pro-tem-we-might-change-our-minds approval that takes some of the pressure off to decide once and for all.
What I'd like to see is (1) evaluation of these efforts, to see if they are working as intended and are free of malignant unexpected side effects, and (2) creative thought in terms of what additional initiatives we can imagine that would experiment with different ways to approve drugs, so we can inch our way towards something that works better, but without taking some bold leap that might end in utter disaster.
Also if you look at the timeline I’m having trouble seeing any individual FDA decisions that actually caused this. Maybe something like ... the FDA makes it too frictiony to get a drug approved, and that encouraged the doctors to be slow in approving the drug and slow in encouraging the fish lipid company to move to the US / any new companies from getting a competing fish lipid approved? Because that feels VERY different from “the FDA intentionally dragged its feet and didn’t approve the drug for 15 years” whereas the treatment developers account has literally everyone dragging their feet and the FDA being helpful. It’s really hard to figure out what sort of changes you’d want when the issue is “systemic slowness and lack of interest in new drugs caused indirectly by a bunch of stuff including the FDA’s slowness in all areas”
Yes, I agree. In this follow-up post the FDA actually comes across looking like one of the few good guys, which is pretty much the complete opposite of the role Scott was suggesting for them by this very example in the previous post. It's a remarkable change, from black to white hat, which is why I think it was awesome and fascinating that Scott wrote the follow-up, and why I think it's a cautionary tale in judging too quickly.
https://mobile.twitter.com/RichardHanania/status/1423141770494234624 “ A bunch of babies kept dying and getting permanent neurological damage, and everyone knew exactly how to stop it, but if anyone did the FDA would take away their licenses and shut them down.” In a sane world the entire FDA would be at The Hague.”
> After more research, I’ve concluded that the story I told is basically true, although I got a few details wrong
I’m not sure that that sentence Richard highlighted was “basically true”? I could be misreading something but it doesn’t feel right. Caleb for instance did link the correction but Richard hasn’t yet
Obviously Scott is much less bad here than anyone else and everyone else with any popularity does much worse daily
Another note is that a lot of anti fda cost benefit analysis claims are of the form “if approval based prescription were relaxed, many of the hospitals who were prevented from prescribing the drug would use it and many of the babies who died from the lipid wouldn’t have”. But if this relies on the fda making things harder to realize drugs are good, leading to the entire medical system rejecting it, that’s harder to argue will just disappear if FDA stuff relaxes. In particular, in medicine pre FDA in the medical Wild West, I still remember a lot of instances of “new treatment worked really well but it still took ten years to convince people” so one might need to target that specifically? But also ensure you don’t get Twitter hydroxyvermectin along with it. Maybe that’s part of why they were so resistant - so many bad treatment ideas come along you stop listening to good ones? Idk.
I do want to understand how that stuff happens. Like there were a bunch of physicians who really thought that their prescribing those were saving lives, and that they were seeing the lives being saved in the patients they were caring for, who they saw. Is it just a random effect thing where one in N attempt*physician combinations will produce a differential in outcome, and then those take off?
Well, that's the Internet. I'm reminded of a great Sidney Harris cartoon, in which one management suit is showing off the brand new (big iron) computer to another: "And in 1/10,000 of a second, it can compound the programmer's error by a factor of 1 million!"
Hard to change when the institutional default is to hold informed consent in such low regard. No matter how much information we give a person, their consent isn't meaningful. Also, we can't compensate them meaningfully for their participation because, again, their viewed as having the same autonomy as insects. (They couldn't possibly resist the temptation provided by an amount of money that could actually have a palpable effect on their life!)
I want to question this: "Discoveries are made by weird tinkering"
There has definitely been a time when that was true. But I'm not sure it is any more. I'm also not sure that you can describe doing something that may well result in the death of a patient as weird tinkering. Tinkering sounds consequence-free, but medical experiments generally aren't. And when your tinkering require Harvard-level education... I'm not sure it's tinkering any more.
I love the lone tinkerer model of innovation as much as the next person, but I don't believe it....
Why? I think this comes out of my professional experience as much as anything. I'm a translator, which is something that millions of people do on an amateur basis. And those amateurs are not creative. I invent more new translations before breakfast than my tinkerer friends do in a lifetime.
So, from (1) a consequences point of view and (2) an inputs point of view, I think professionalism is a better model for creativity than tinkering.
An aspect Scott didn't go into in much detail is, why should it be this complicated and difficult for credentialed doctors to use a medicine or supplement that is used, apparently without problems, by millions of people per year? Sure, maybe the FDA wants to have its own particular standards of scientific proof before officially announcing that a medicine is safe and effective. But why **forbid** a doctor to try something only very likely to be safe, rather than extremely, if they judge that it's worth trying? After all, we already allowed this in all sorts of degrees. Doctors routinely prescribe precise schedules of dosage that haven't been explicitly tried before in an FDA approved study. They routinely prescribe combinations of drugs that haven't been evaluated in combination in an official FDA study. They also frequently prescribe drugs for uses that were never specifically evaluated by the FDA. There is some gray area here around how officially sanctioned these things are, but they are certainly not actively blocked by the FDA. So why block prescribing or importing a medicine that has been evaluated in trials with thousands of participants, by another agency that the FDA has general trust in? Wouldn't that still be an extremely high standard of evidence?
I have no idea as to the internal workings, but I do wonder if there's some pressure from the pharma companies? "We have to spend billions to test to get certified but you're letting some foreign company drug onto the market here for nothing? And they get handed our market for this illness on a silver platter? We will not stand for that!"
At the bottom of the FDA story, there's the problem of how to combine the careful approach needed to test new drugs with the somewhat reckless approach needed to do the cowboy thing. In other words, how to give researchers some slack without compromising the safety provided by the existing process.
Now, I've been a Google SRE for several years. These are the people who make google so reliable that you do ping www.google.com to check whether your network connection is working. The problem there is similar: You want a rigorous process to make sure that stupid mistakes won't happen. At the same time, you want those people to act like cowboys when the shit hits the fan.
One part of solution is the so called "blameless culture", i.e. if you screw up, you are not blamed. You won't be blamed by your colleagues or higher-ups. Your name won't leak out of Google. It will be Google taking blame for the outage, not you as a person. That gives you the slack needed to do the right thing when needed. But the blamelessness culture is complemented by, let's call it a "postmortem culture", that is, the process of incorporating what you've learned in the cowboy mode back into the formal process. It involves writing down what happened that needed the cowboy response and all the interested parties discussing how to prevent such scenarios in the future. The third component is having people on the ground deciding on the changes to the formal process, no political supervision involved.
Couple of comments:
1. The blamelessness principle also exists in politics. Some conutries, e.g. Germany, Switzerland, EU, use so called collegiality principle, where the ruling body makes collective decisions and the blame for the outcomes as a body, not as individuals. This also has some nice side effects, like putting the members of the body on the same boat and making them more prone to cooperation than to fighting each other.
3. No political supervision for process changes may sound hard to achieve in the real world, but once you give people on the ground the right to go cowboy anyway, the process becomes more of a "best practices" thing and there's little political gain in trying to influence it.
Book review request: Upton Sinclair's "The Jungle" I'd be interested to see whether there's a reason for the FDA's madness to be found in its origins, (and also to see Scott grapple with what I can only imagine is the Gilded Age equivalent of a quirky rationalist blogger).
Sinclair would be a tanky, I think, not a rationalist. (Less inflammatory, he was a leftist and was trying for more regulation and bigger, more emotionally driven government regulation.)
Does anyone know why FDA approval costs so much? What components of the process add up to a billion dollars? We need ideas which reduce the whole cost of the pipeline without compromising standards, starting with understanding where the current costs come from.
Not my field, but my vague understanding is (1) clinical trials, in particular the huge Phase III trials needed to prove efficacy, are very expensive, and (2) the cost of FDA approval is generally calculated by dividing the total research costs for all the drug candidates divided by the number that get approved.
Since almost all drug candidates fail at some stage of the process, you have to add up the cost to synthesize 10,000 early candidates, the cost to optimize 1,000 promising molecules, the cost to do animal studies on 100, the cost to do Phase I trials on 10, the cost to do Phase II and III trials on 5, and attribute all that monster cost to the 1 drug that makes it all the way through the gauntlet and gets approved. (Those numbers are totally made up, the reality is probably if anything worse.)
One way to address (1) would be to have the government pay for Phase III trials, i.e. move the subsidization from all the future patients who use the successful drugs to all the taxpayers, on the grounds that we all benefit from this kind of research. It would certainly "reduce" the outrageous cost of on-patent successful drugs, but not really, it would just shift the tab from patients who use the drug to everyone's tax bill (although it would obviously be much less per capita). But there are lots of complications there, naturally. It would definitely shift the motivations of drug companies, if they weren't risking their *own* money on Phase III trials, but whether for good or ill is hard to guess.
I'm not sure there's any good way to address (2) other than to have drug candidates fail faster and earlier, and the pharma industry already tries their best to do that, with all kinds of tricks. And you don't want to do that *too* much because then you discourage genuine innovation, and everyone spends his time doing the pharma research equivalent of making "Iron Man XVIII", an endless series of knockoffs and reruns, instead of trying something risky and new.
What if instead of the 1 successful drug pays the monster cost of 10000 candidates, 9999 failed candidates pay the 10000 in total? What you described seems like promoting spamming as many candidates as possible since you don't pay if it's not approved?
Did I misunderstood something? English is not my primary language.
The pharma companies do pay those costs. But any drug that gets through has to pay back all those costs for the company to break even. This isn’t approval cost as in “company pay the FDA all this money”, but costs as in “this is how much money a company spends on all of drug development on average per approved drug”
The successful drug is the only one that can be sold, and therefore the only drug that actually produces revenue. So its sales need to pay for *everything* the company does, including all the failed drugs.
Imagine cars had to be designed by trial-and-error, and 99 times out of 100, they would explode the first time they were driven -- *and* there was absolutely no way to know whether they would explode except by building them and trying to drive them.
So Ford pays a big team of engineers to design new cars, and unfortunately 99 times out of 100 the new cars they design explode and can't be sold, but they spend a lot of time designing and building the cars. 1 time out of 100 the car *doesn't* explode on testing, and then they can sell it. But the revenue they get from selling that 1 model of car has to pay for the costs to develop the other 99 that exploded as well. So it's a very expensive car.
I don't know if this is discussed elsewhere, I haven't read all comments but enabling cross approval with EU seems like it would at least alleviate many of the problems. In aviation for example, up until the 737Max catastrophe, EASA (European equivalent of Federal Aviation Administration) recognized approvals by FAA (and I think vice versa). By the way, there are interesting contrasts between how FAA makes life-and-death-decisions-that-has-insurance-consequences and how FDA does so. FAA, probably due to budget cuts or something, and probably also because airplanes are getting more complicated, doesn't have enough inspecting engineers to license an airplane, so it asks the company in question to kindly license the airplane they developed. This sounds scary, and it is.
Maybe it's because they don't worry as much about losing public trust? I don't know but I believe this is something to think about.
My main point was though the cross-approval between US and EU. I probably wouldn't do this with an economy which US is in a kind of economical cold war with such as China though.
The FDA culture still remembers thalidomide, which was approved in Europe and the UK around 1960, and caused something like 2000 infant deaths and 10,000 birth defects before it was pulled from the market. The famous Frances Kelsey as an FDA reviewer declined to approve the application for it to be marketed in the US, based in part on the fact that it was prescribed as a treatment for pregnant women *but* no studies had been done on its effect on the fetus. As a consequence of her intransigence in the face of "but 40 other countries have approved it!" the United States was almost entirely spared the consequences of thalidomide use.
EU and UK approval bodies probably have learnt from their mistakes in the last 6 decades, and even if not, the amount of lives and life quality lost now seems more than what would be lost if they miss a Thalidomide once every century or so based on how Scott writes about FDA.
They probably have, but the only way to know for sure would be to fling the gates open and see if another thalidomide happens, and I would guess they don't see any compelling reason to do that.
If European drug discovery and approval were twice as fast as the US or something, and the US was lagging seriously behind in the latest innovations, then there *would* be some kind of solid reason. But so far as I know, the general pace of approval is not that dissimilar -- if anything, it's a bit faster and earlier in the US. A quick gpogle search suggests the FDA approved about 60 new drugs in 2018 and the EMA about 40. It also appears that of the drugs approved by both the FDA and EMA, about 2/3 are approved *first* by the FDA. So it doesn't really look like in general the FDA is dragging its feet relative to the EMA, even if for this drug or that there is some discrepancy that is obnoxious to patients.
And, with all due respect to Scott, his is an amateur analysis, so while it's interesting it's only a starting point, a better analysis might not reveal the same thing. Furthermore, the question isn't just one of numerical analysis: whether it is "better" to save more lives through easier approval at the cost of the rare disaster is an ethical judgment, and reasonable men may disagree on the answer.
For what it's worth, to the extent we can discern the dominant consensus of ethical values among Americans, it does not seem especially congruent with the utilitarian calculus that just reduces it all to QALYs saved and lost.
" FAA, probably due to budget cuts or something.. asks the company in question to kindly license the airplane they developed. This sounds scary, and it is." -
An alternative to having a company review its own product is the "reviewer of reviewers" model that is used in the EU. In the EU the EMA contracts out some of the reviewing to private firms. This introduces competition and incentives to improve things into the system. I saw a paper analyzing this that I think was written by someone who is involved with Effective Altruism, but I can't remember who the author was and can't find it now! I think it was a thesis of some sort. If I stumble upon it again I'll post it here.
In this case, because the placebo group would have several dead babies that would live on to long and happy lives if they weren’t placebos. (Assuming whatever treatment it is works, which it does here
That was my initial reaction too. But if the trial is run with a placebo group it is more highly regarded, the treatment may be approved and there may be many less dead babies in the future.
This is were Utilitarianism becomes ugly. Solving for the least number of dead babies may be optimal, but at the same time it is horrible
If babies die because the standard of care is bad, that's just how things are. If you call those same babies a 'control group' you suddenly have a duty of care.
If hundreds of Germans die because there weren't enough covid vaccines, what can you do? If one dies of a blood clot after getting AZ, then the medical system has failed them.
I propose the Alcohol Standard: Anything less dangerous than alcohol should be legal.
If you want to make a drug legal, all you should have to do is show that it is less dangerous than alcohol. If you want it to get FDA Approved or if you want to make a drug which is more dangerous than alcohol legal, then a more complicated process should be required.
I was a pediatric intern in 2013 and can offer a small amount of insight into how TPN cholestasis and Omegaven was viewed in NICUs from 2012 when I first encountered discussions of it through to now.
As background, the TPN (total parenteral nutrition – the nutrition delivered into veins) cholestasis (lack of bile flow) has been a described complication of TPN or PPN (partial parenteral nutrition) for a very long time. It afflicts neonates who aren't able to take a sufficient amount (or potentially any) nutrients via their GI tract. It also affects older children (and some teens and adults) whose GI tracts are unusable or those who have so called "short gut syndrome" (often children who had significant portions of their small intestines removed due to an infection called necrotizing enterocolitis that is not uncommon among significantly premature infants).
In the early 2010s it was widely known that some small percentage of patients treated with TPN would go on to develop cholestasis, which was an easy thing to check by monitoring the patients' bilirubin levels. About the only identified risk factor for who would develop it was how long the patient had been solely receiving TPN.
If/when they developed TPN cholestasis there were a few options depending on the patient circumstance. In some cases you could decrease the amount of calories/day that the patient was receiving from the lipid emulsion and their cholestasis would improve. In other cases you could stop giving them the lipid emulsion component of TPN (TPN is a very complex product that has a fascinating history, it includes carbohydrates, proteins, trace minerals, vitamins – everything you need to grow and survive) for a period of days-to-weeks and then reintroduce it and often times their cholestasis would not immediately recur. In other cases it would, which usually meant that a pile of paperwork got filled out and even back in 2013-2014 you could get them compassionate use of Omegaven, which the FDA acknowledged seemed not to cause these problems. While the Omegaven saga was playing out, in fact another product was undergoing active study and trials in the US what was an option for patients to be enrolled in initially, or to also get via compassionate use – that product was SMOF lipid, which is used much more widely today than Omegaven is, and getting patients who'd failed the traditional soy lipid formulation onto it was the same paperwork but reliably effective.
The fact of the matter is that even today, in 2021 with fully approved intralipid (the old "problematic" lipid formulation, Omegaven, and SMOF-lipid) many hospitals in many circumstances still use the original intralipid formulation because it is not uniformly problematic, the problems are easy to detect in the initial stages (and they're reversible!), and the alternatives are significantly more expensive.
I'm an enormous critic of the FDA and many of the ways that it undermines appropriate research on a variety of things by giving out too many compassionate use waivers and wastes valuable time on time sensitive investigations, but the story of intravenous lipid preparations is not one of FDA failure. There were many failings of the wider medical establishment in recognizing and propelling Omegaven forward with more alacrity, but that is a totally different problem of insularity and arrogance.
"it undermines appropriate research on a variety of things by giving out too many compassionate use waivers"
Yes, but as a public body it is very susceptible to pressure when interested parties are feeding the media "Babies are DYING" stories. Desperate patients and their families, cowboy medical professionals and drug companies all collaborating to lobby for compassionate use waivers are hard to resist, and even if you are fully prepared to be a stony-hearted bureaucrat swathed in red tape, your political masters are not when a canny grass-roots campaign to "contact the local congressperson and ask them to sign on to this!" is in place, and Senator Braces who is the representative of the Desperate Families in question starts making noises at the administration to tell them to tell you to let this drug through.
Reading this made me realize I really don't know anything about how medical studies and regulations around them work. Why was it never a relevant question to go "Hey, do the babies over in Europe who receive Omegaven get PNALD at a lower rate?" Shouldn't that be significant enough to get you out of the "rut" of "drug is too good to make a control group justifiable, but doesn't have enough evidence behind it for people to care."
I have some understanding that the FDA prefers (or requires?) American studies, or something, so maybe this is the problem?
I'm pretty sympathetic to the need for "cowboys" in medicine, because those are the kinds of doctors who actually got transition medical care for trans people to the semi-tolerable point where it is today in some countries. It took some surprising guts and a certain amount of generational change for doctors to actually start listening to what their trans patients were saying. And improvements in HRT protocols are all done pretty "cowboy" because of the difficulties of getting any decent study funded and all the drugs involved being used off-label. The official guidelines from groups like WPATH or my local org Rainbow Health Ontario, while less bad than what was done two or three decades ago, are still pretty suboptimal in a lot of ways based on some of the preliminary stuff that comes out of clinics which are willing to experiment a bit. If you get progesterone as a trans woman (which seems to help complete breast development, improve libido, etc.), you have to sign a waiver which says "officially this does nothing, but we agree to prescribe it to you anyway", and many doctors just won't prescribe it because officially it does nothing. There's also a bunch of misinformation about the risks because the official guidance doesn't distinguish between different types of progesterone medications, some of which have significantly worse side effects than others. Repeat for a variety of other medications and combinations thereof. You need someone willing to do their own research and push the boundaries in order to get the best care.
At the same time, it's a bit of a double-edged sword. One such cowboy in the UK was trying experimental psychotherapy of dubious validity (specifically, therapy intended to make a gender-dysphoric person not transition, which has little evidence to support its effectiveness and a good amount of evidence to show that it actually causes harm, as I hope you know) on non-consenting children who called a mental health helpline about their gender troubles.
This is also why I have some fearful uncertainty about some of your suggested changes to FDA approval and drug coverage in a previous post. Because transition medications and other procedures will basically never get a good randomized controlled trial done for it for a variety of reasons, it would be hard for it to get the kind of full approval needed to force insurance coverage in your suggested system, even with cost barriers to getting certification removed, which leads to a big mess of coverage or lack thereof (coverage is already a big mess, I foresee this making it worse) and it being used by red states to score culture war points, as trans care for minors currently is.
Am I the only person who finds the statement 'Every single one of the parents of a child in the control group noticed such inferior results that they begged and pleaded with us to let them have the treatment and we being softies gave it to them so our results are weak' more than a little bit insane? Like, surely whatever early data the parents were noticing was so strong that you could have recorded it and used that as the basis for robust results? Then again, there is the catch-22 of not letting anyone do a study to prove that it's effective because it's too effective for it to be ethical to have a control group, so there's plenty of insanity to go around here.
I'm amazed that nobody has mentioned the controlled trials on the use of oxygen in premature babies. It's my understanding that the doctor who did the study knew that he was going to blind some of his subjects and possibly kill a lot of them, but that has prevented a lot of babies from being blinded since then. I don't know what it's ethical to do, but in cases where the accepted treatment is worse than doing nothing and it's almost universally used, one can make a strong case for acting.
I appreciate the deep-dive post. I actually came away from this thinking that it somewhat weakens the overall argument that the FDA is in the “wrong” in this case.
> I realize the hurdle is there for a reason.
I’d be really interested to see you flesh this out further.
I think we are covering a subject where it’s just not valid to look only at the costs of the current regime. We need to also make an effort to compare the benefits too. A zero-regulation environment (or even just a move directionally towards less regulation / faster approval) would clearly also involve harms, or put differently, the status quo is averting real harms. The question is whether for a new position on the spectrum, the new benefits would outweigh the new harms.
Thalidomide is the obvious one that the FDA remembers deeply, but I think it would be more interesting to (say) investigate the covid vaccine candidates that didn’t get through the approval pipeline, or some other drugs that were dropped at phase 3/4 due to side effects that were only detectable with very large studies. Essentially, try a steelman of the FDA’s current level of conservativity based on the public record.
Another related critique I’d make of this piece is that the drug approval process is complex enough that I’d be very suspicious of claims that you can make a localized change to fix one small scenario like “allow Omegaven to be used when it clearly works”, without having substantial knock-on effects on potential widely-used drugs that would swamp the positive utility of fixing this case. For example, would we also get statins which are more efficacious but we later discover have bad side effects? My impression from conversations with people in the field (so a low-confidence assertion) is that there are lots of drugs where the cost/benefit ratios are not that far below 1, and so undetected side effects would take it from a net benefit to a net harm. Would be interested in others’ thoughts/input on this angle. To be clear it does seem to me that there are improvements to be had, but I think we’re optimizing a U-shaped function and need to be careful not to make things worse.
There are two distinct but related aspects of human nature that make this problem fraught:
(1) Human beings are by nature optimistic[1], meaning they will generally overestimate the probability of new approaches being successful. If you leave it up to "common sense" and intuition -- even that of experts -- you will generally considerably raise the level of garbage relative to gold, if history is any guide often to the point where it becomes almost impossible to discern the signal from the noise.
(2) Human beings usually prefer to rationalize their failures than admit them. It's very difficult to say "well, I was just wrong" instead of "well, I was right -- but not at the right time/except for this weird exception/but my idea was executed properly/in principle." This is most especially true when the stakes were very high, e.g. a human life. It's extrremely difficult to say "I was wrong and someone (maybe me) died as a result." People will struggle to find *any* rationalization for why they were actually right, but some other random factor prevented their rightness from resulting in success.
Both these factors mean that medicine as a field of human knowledge has advanced shockingly slowly throughout human history despite its overwhelming importance to us. It has been held back for years, decades, centuries, by inherent optimism about one's own hypotheses, the inability to accept failure, and our inherent preference for rationalization instead.
Presumably the fact that stakes are so high is why these psychological and sociological factors have held back medicine much more than they held back, say, the development of steam engines or electronics. If you build an engine based on Theory X and it blows up, it's just a lot easier to accept your failure and learn something than if you treat 10,000 patients with Theory X and they die or suffer horribly. One would guess that is why historical events we now find horrifying -- trepanation or leuctomy for mental disease, bleeding and cupping, the failure to realize plague was carried by fleas on rats, the failure to appreciate that clean hands saved lives during operations -- went on for so long.
The post-Enlightenment solution has been to force ourselves to act like skeptical empiricists, even though it is very much not in our nature, and nobody does it naturally -- it's like being on a constant mental diet of raw broccoli and bran muffins, never allowing yourself the pleasure of the ice cream sundae of just believing what your instincts suggest is true. We insist everything, even the stuff that seems obvious, must be tested in nitpicky and even OCD detail, and we take extreme safeguards against optimism and rationalization having a role in final judgment, however fraidy-cat unimaginative heartless absurd that may intuitively assume. We try to institutionalize Murphy's Law.
The practical results are certainly impressive: the world has seen advances in science and medicine since this became the scientific norm that utterly dwarf anything accomplished in the past 40,000 years, with perhaps the exception of the invention of speech (and we did not accomplish that through conscious effort anyway, one assumes).
But we are perfectly capable of rationalizing away that success, just as people can rationalize that their weight loss and improved health came from happier thoughts from a new guru, or a new arm workout regime, rather than all the damn broccoli and bran muffins and absence of ice cream (and then abandon the bran and gain back the 15 pounds promptly, which will in turn be rationalized as the result of sleeping on the wrong pillow or not enough cinnamon).
It has been so long since we lived in the squalid world of scholasticism and alchemy that I daresay in the West we are capable of restlessly abandoning the unnatural and unpleasant strictures of Western empiricism, and taking our heritage of technological progress entirely for granted -- treating it as some kind of natural order of things, instead of a hard-won victory over our natural tendencies. That *would* be in the nature of humanity: as they say, soft times lead to soft men.
Huge fan, but I think Scott's being too easy on himself here. This was THE example that the rest of his initial FDA takedown hung on - the implication was "throw a dart at the FDA and here's what you'll find, it represents a thousand other stories just like it." It turned out to be substantially wrong. Like, AT LEAST as factually wrong than the crappy NYT story about Scott which he (correctly) didn't excuse.
Scott's charging for content now, and is doing important, high profile investigatory journalism. It's not OK to get so many relevant facts wrong. "This story is directionally and emotionally accurate, so don't worry that my supporting arguments were wrong" is exactly the kind of BS I want to see ACT/SSC as fighting against in the rest of media.
I would've liked to see a full retraction here, and most importantly, I'd like to see some of the subscriber money go to an intern who can help fact-check.
(and again, I'm a huge fan -- that I can even imagine Scott taking the high road here is the reason this disappoints me so much more than the typical BS in another publication.)
Okay, it's true. We'll currently fast-track applications for drugs that were approved by the British NHS, and it's already considered one of the easier ways to enter the American market ("apply in Europe first"), but that fast track could easily be an automatic reciprocal approval process among a treaty of nations (Canada, the UK, the EU, I would accept safety data from Singapore as Phase 1 trial data here though their efficacy data is all postmarketing studies). PS: I feel like the flip side of the FDA being "default no with high hurdles" on drugs is this story about the FDA continuing to permit the implantation of medical devices as evidence mounted against their safety https://www.propublica.org/article/heartware-patients-implanted-fda (and also 510(k) abuses, with the history of the vaginal mesh catastrophe, the cobalt-laden joint replacements causing bone degradation, the heart valves that caused clotting and death; those are just the ones I remember off the top of my head).
Who cares if it's reciprocal? If those other nations want to waste money re-checking already safe enough drugs, they are welcome to do so. Framing it as a treaty is just counter-productive, it can and should be a unilateral decision.
Another typo: "... the got ..."
In: "They asked the FDA for permission, filled out the relevant forms, and (to its credit) the got approved within 48 hours."
No. Scott is contrasting with the 10% * 10% * 10% * 10% that precedes it.
Typo: "...in this country. briefly..."
Since you're probably going to see this comment - I have a random question: I'm using the sun lamp for 30 minutes when I wake up, at 9 AM. If I instead wake up at 9:30 or 10 or 11, should I do it then instead, or skip those days? Thanks!
Speaking as a sufferer from SAD who has used light boxes in the pas (although rarely these days, as I have found it more effective to adjust my lifestyle to have frequent walks in the sunshine whenever possible):
I would not worry about the exact timing too much. It's not really that finicky and precise of a thing. Remember it's just artificial sunshine; so if you wouldn't be afraid to go to the park on a sunny day at 11 am, you definitely shouldn't be scared to use the sun lamp at 11 am!
In my personal experience light therapy is moderately effective to fight depression, when taken at any time of day. The main reason to do it in the morning is to prevent it from keeping you awake too late at night, because it resets your circadian rhythm.
Think of light therapy like a dose of caffeine---if you take it in the late afternoon or evening, you might end up staying up a lot later than you intend, or sleeping poorly (though unlike caffeine, using the light box early enough in the day will probably actively help you to sleep better in the night). But you probably wouldn't angst about taking your cup of coffee at the exact same hour every morning, unless it is really important for you to have a totally predictable schedule. (This seems not the case for you, if you regularly allow yourself to sleep in.)
You can even use your sun lamp in the late evening if you are having a really severe depressive funk and care more about ending it than going to bed at a reasonable hour (but be aware that getting good sleep is also helpful for treating depression). Or more reasonably, you could turn it on for an hour at 4 pm if you are in a place where the sun sets that early in the wintertime, and you wish it hadn't done that. (Maybe in that case it would be better to use it more as ambient light cheer rather than spending a long time staring into it.)
The risks of light therapy are pretty low, so there is no harm in experimenting and finding out what works for you. So try using it at 11 am, and if it doesn't cause any noticable problems for you, then you're good!
Oh, I'm just trying it for circadian rhythm reasons, not depression. Thanks for the comment!
"I can’t find good sources about whether the issue is that soybean oil *prevents* the liver problem, or that fish oil solves the liver problem."
There is a word wrong in this sentence, or I fail comprehension. I think "prevents" was intended to be "causes", but I'm not sure.
And, as long as I'm being super pedantic, it's "pejorative", not "perjorative".
Thanks, fixed.
"...decided to do the reasonable thing and walk my patient through the process of ordering it illegally on the Internet."
While this is very cool and right, I'm concerned about the advisability of admitting it publicly
I find it unlikely that explaining to someone that the internet can be used to do something illegal is itself illegal. Then again, I don't know what weird laws the US might have...
I suspect the details are variable based on a whole host of factors know one is quite sure of? After all, there are (or at least have been) states where cannabis is illegal, including medicinally, doctors can't prescribe it, but they are allowed to recommend it.
Many of our weird laws are there to give authorities enough rope to hang anyone they deem hang-worthy. Actually trying to follow all of them would make you feel like Jamie Lannister complaining about how many vows knights have to swear.
Doing it in your capacity as a doctor seems potentially career-ending. Doctors can lose their license without doing anything illegal.
This is a side issue, but there really needs to be a single word or short phrase for "we are pretty sure this medical practice works, the existing controlled trials are not good enough, we will never be allowed to conduct the trial because no one will let us have control group". Examples:
- Diphtheria antitoxin in the early 1900s. There is a section in the book The Microbe Hunters about how the initial trial had lower than usual mortality but no control group, but now (written in 1926) we'd never refuse to give a child antitoxin so we can never do a controlled trial. So like, we're pretty sure this works but we'll never know.
- Surgical masks to block spread of disease. See Scott's posts back in March 2020.
- Apparently, Omegaven rather than soy-based competitor.
I suspect there's many others, even excluding obvious ones like "bandage wounds instead of letting people bleed to death".
How about "the parachute principle" after the famous lack of RCTs on parachutes?
e.g. "this is a parachute drug" or "this is a parachute intervention" or something like that.
I like this phrasing.
Genius!
If everyone is convinced that leeches work, then clearly it would be unethical to not use leeches, so we can't ever gather the evidence we need to stop using leeches, unless we get lucky with some kind of natural experiment where the hospital's shipment of leeches got lost in a hurricane. I think from a utilitarian perspective it is almost always worth it to have a control group even if you have a strong prior for the treatment being better than the control. Especially if the consequence of not having a control group is that it will continue to be illegal for almost anyone to use the treatment. The control group is no worse off than they would be if they weren't in the study, so go ahead and have a soybean oil based control group.
The difference is that leeches, in your hypothesis, are an ineffective treatment that is already deployed across the entire patient population. The picture is very different if the drug or procedure we want to test is something newly-introduced: in cases with massive and obvious effects, the *past* can serve as the "control group" showing what happens with patients who don't get the new drug.
Yes, but does the FDA ever approve things based on massive, obvious improvements compared to the past SoC, without a real RCT?
Well, no, and that's the problem.
I recall there was some doctor in New York with a case series of HCQ on covid patients that looked like a massive, obvious improvement to the SoC, but then people did RCTs and it didn't work at all. So at least sometimes the biases of a non-randomized sample can make it look like a big effect when it's not.
Isn’t this post about the FDA doing just that?
You could easily do RCTs of parachutes on inanimate objects or squirrels (without killing the squirrels!)
Hardly anything in medicine is as simple as the physics of terminal velocity, though. You probably can't work from first principles and get as strong of a prior for any medical treatment as you'd have for a parachute. Personally I wouldn't trust any drug without an RCT. _maybe_ I'd trust mechanical ventilation without an RCT, but that's iffy (what about damage to the lungs, circumventing all the homeostatic mechanisms for CO2 levels that control the PH of the blood, and doctors getting overly paranoid about SpO2 of 93% when it's not a real problem until SpO2 drops below 85%? How do we know what is the correct SpO2 level to make mechanical ventilation worth the risks, anyway?)
I mean you *could* do an RCT on parachutes on inanimate objects, but why would you need to randomize them?
Assumedly to blind the experimenters so they don’t push the things out of the plane differently or something.
I was assuming you'd drop them off something high e.g. a crane, like they do when testing parachutes on Mythbusters, since it's less expensive and more controlled than a drop from an airplane. But yeah maybe the airplane itself introduces confounding factors that need to be considered for a real world efficacy trial for parachutes...
Also, you're talking about safety trials, but in the Omegaven case what was (and apparently still is) missing is an efficacy trial. In other words, Omegaven has been given to loads of babies safely in Europe and there isn't really any doubt about its safety. The problem we have is that we don't know exactly how effective it is because no one is willing to run a study in which they know some of the babies are going to be denied a parachute.
Going by this page, there were 75 studies done on Omegaven, and only 8 of them so far are completed with results:
https://clinicaltrials.gov/ct2/results?intr=Omegaven&recrs=e&age_v=&gndr=&type=&rslt=With&Search=Apply
Several of these are "compassionate use" so they have tiny numbers (e.g. 2 infants) and there's not really enough data to say 'a resounding yes for Omegaven'. It seems to be helping, but even so some adverse effects are recorded.
Looking at those 8, they're all open label, single group assignment studies - in other words, neither randomized nor controlled.
Good news, I just performed an RCT of parachutes on squirrels, and both groups were fine. I even did a second RCT on mice and both groups were fine. Now it's time to move onto human trials. We're recruiting volunteers, would you be willing to sign up?
:)
Even in the medical context, lots of drugs have worked on mice but not in humans or vice-versa. At some point we either have to do an RCT on humans or we have to assume that they'll work on humans when they worked on the non-human trial.
We've been monitoring this biomarker called "terminal velocity" across species, and it looks like as the size of the species goes up, the terminal velocity also goes up. If we extrapolate that line out to humans, they'll probably go splat. But our parachutes were able to reduce terminal velocity by 95% in all the animal trials, so let's go ahead and try one on a human :)
Squirrels have a terminal velocity of 40km/h, but you can't scale that up to humans. By that nonsensical logic, a blue whale would have a terminal velocity of 700km/h! This is clearly nonsense. Let me write a longform article about how your field is filled with fools and needs to divest itself of the "terminal velocity fallacy" in order to be taken seriously.
I think you forgot that you were discussing control groups.
I suffer of a chronic disease without any real cure. So quite interested in research. I stop reading studies when it turns out it's a mice study (some are amusing though, like Viagra curing good knows what...)
Viagra was originally heart medication, with erections as a side-effect. It wouldn't surprise me that increasing blood flow generally could be beneficial for all sorts of conditions, particularly anything that's more like wound healing than fighting off an infection.
Today, only drugs proven safe and effective are FDA legal.
A simple reform would be to legalize drugs when proven safe.
I agree. People might object that this would enable snake oil salesmen, but people are already allowed to sell unlimited snake oil in the US, without even proving safety, so long as:
1. they call it a "dietary supplement"
2. they use weasel words to describe its effects
3. the substance can be found somewhere in nature
4. the substance has never been investigated as a potential drug.
None of those criteria have anything to do with consumer protection. It would make sense to replace all of them with "just prove it's safe".
N-Acetyl-Cysteine just got taken off the market after half a century of safe use because someone at the FDA noticed that someone was investigating it as a potential drug in 1963.
Isn't acetylcysteine simply a drug in the US? It's sold as a OTC medication – not a supplement – where I live.
That's not exactly how it works. There is a large number of "general recognized as safe" (GRAS) ingredients for which no FDA approval is required, roughly because they have been consumed for a long time and nobody has yet noticed anything evil. (Whether the ingredients are "found in nature" is neither here nor there -- plenty of genuine drugs, e.g. taxol or penicillin, are natural products -- it's a question of whether loads of people have actually been eating it for decades.)
That's why you can add salt or gelatin or vitamin C to a supplement for which you claim health benefits but *not* have to submit a drug application. People were eating those ingredients for decades and even centuries before the FDA was formed, and nothing bad was noticed, so it makes little sense to attempt to regulate them now. Although obviously that does *not* guarantee they're harmless, cf. cigarette smoke, in use since the 1600s. Nevertheless, the FDA's ambit by statute doesn't include GRAS ingredients, so caveat emptor and away you go.
You sure do have to include weasel words in your advertisements if you want to claim specific health benefits. You can make generic claims like "good for you" or "healthy!" all you want, but if you say something specific like "lowers your blood pressure" or "cures cancer" then you do need to add a specific disclaimer that says you haven't asked the FDA to evaluate your claims, because people might naively otherwise assume you wouldn't make such claims unless some massive RCT study had been done and the results certified by the FDA. (The fact that this annoys the neutraceutical and alt-medicine Laetrile aficionados is to my mind evidence that they indeed would like to free-ride on the reputation of real pharmaceuticals for rigorous testing. If they thought FDA approval meant nothing to their customers, or wouldn't be otherwise assumed, they wouldn't give a damn about adding the disclaimer.) But having to add disclaimers is separate from being able to use GRAS ingredients.
NAC got taken off the market because by statute any ingredient which first appears in a consumer product as an FDA-approved drug cannot ipso facto be GRAS. In this case, NAC was approved as a drug in the 1960s, before it was used as a supplement, so it can't be GRAS. That the FDA hasn't rigorously enforced any restriction of it until recently doesn't change its legal status, there's no such thing as a "legal easement" where if you get away with a violation of the law long enough, or it's small enough, it becomes technically legal.
Someone could argue NAC should go back on the GRAS list, and I don't doubt many someones have or will, but the way to do that is do the right (expensive) studies and offer persuasive factual evidence, not make an emotional appeal to intuition: "everybody's been doing this for years!" -- yeah, well, people smoked for centuries before someone figured out it gives you lung cancer and heart disease, which is kind of bad.
And there are a ton of randomized trials of parenteral soy vs fish oil in mice!
Almost nothing meets this criterion, though. https://www.cmajopen.ca/content/6/1/E31
There are several RCTs and at least 25 published trials of soy vs something including fish, so you are right in this case
This article is fantastic. I also note that it uses the "parachute" terminology already - in particular, to argue that many practices which are claimed to be parachutes are not even close to being parachutes.
Honest question - in Science! am I allowed to use natural experiments as a source of data?
EG in 99.9999% (a few famous people lived) all known cases since the dawn of heavier than air flight of falling from great heights and an aircraft in flight *without* a parachute has resulted in death. In our work with parachutes, 99.9999% of our our research subject survived a fall when protected by a parachute avoided death.
Can I use those two collections of data to reason from like a control and not control group? To avoid exactly the silly argument described here that there is no RCT on parachutes?
Yes
Presumably one can and should, in those situations, assemble the evidence one can in the absence of RCTs such as retrospective comparative studies, which still could be persuasive if the effect is large enough (as it would be if ifs a moral imperitave to not do RCTs)
Re surgical masks, here are two studies concluding that masks in surgery are not helpful, based on experiment.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2493952/pdf/annrcse01509-0009.pdf
https://link.springer.com/article/10.1007/BF01658736
I read only the abstract of the second one, it is behind a paywall.
I'm not sure how convincing these studies are, but it's not true that no controlled trials of surgical face masks have been done.
IIRC, breast milk vs the alternatives.
I attended a research talk about mitigating the harm of sepsis by activating an enzyme (Heme Oxygenase 1) naturally occurring in the body. The presenter basically claimed it worked miracles, and i found it convincing. Three years later it has not entered common use.
Something similar happened to me, where the FDA did go faster than their normal process, but still nearly killed me in the process. Dan Elton, of the US Transhumanist Party, wrote an article about it:
https://moreisdifferent.substack.com/p/the-fda-almost-killed-me
See also, a whole lot of people who died of COVID in November-December 2020, while the FDA was evaluating clinical trial data (albeit at "warp speed").
Warp speed is an FDA euphemism meaning "it took us three weeks to schedule a meeting".
Worse than that - scheduling the meeting and getting the right people to look over the reports usually takes much longer. The COVID vaccines got pushed to the front of the line, just like they did for every intermediate approval. Three weeks really is warp speed for the FDA.
Yeah, I know. The sarcasm is biting in every direction - that comment was like a hydra of snark.
What's doubly depressing is we're repeating the exact same mistake with the third-dose booster shots against the Delta variant, and vaccinations for younger children.
Reading your story, it's not "The FDA almost killed me", it's "the lawyers almost killed me" because the pharma company went in for legal wrangling with the FDA over the blackbox warnings which few people ever read:
"Even though the FDA had granted Enyvio Priority Review Status, for the next five months the FDA kept John (and undoubtedly many others) suffering as they debated the wording for the warning label. Since the approval of earlier monoclonal antibody drugs such as Remicade and Humira, scientists learned that in the long term some patients can develop allergic reactions to treatment. The FDA wanted to put a warning about the possibility of a severe allergic reaction on the label. The problem was that the drug had been designed specifically to avoid the allergic reaction rejection problems that occurred with Remicade and Humira, drugs which did not contain such a warning. A warning about allergic reactions would make Entyvio look less safe, something that the company rightly took objection to."
Of course the company objected to it, but you can equally say there "if only the company had agreed at the offset, it wouldn't have taken five months of lawyers exchanging letters to get this done".
And if the FDA had let it go on what the company wanted, no pharma company ever wants "possible side-effects" put on warning labels. I have a dog in this fight as my doctor recommended me a medication, gave me the obligatory warning of possible side-effects, reassured me that such was very unlikely then wrote me a prescription.
I read all the warnings on the box, was highly alarmed by the "this is rare but IF it happens you are in deep doo-doo" warning, looked it up online, yes indeed it was a real, rare, but deep doo-doo side effect. "Well, it's rare, I have little to no chance of getting it" says I; I take my first dose of the medication - and I get the starting warning symptoms.
"Okay, not taking any more of this", I say, I tell my doctor why, she tries to talk me round, I refuse, and I get put on a different medication.
And I do firmly believe that, had it not been for the FDA or similar body sticking it out to put the "it's very, very rare BUT" warning on the label I would have squashed my reservations about the initial symptoms because the doctor says it's okay, continued to take the medication, and ended up in deep doo-doo indeed.
While I see your point, that legal wrangling didn't need to happen at the end of the process - as the story points out, the trials and approval process lasts for *years*. A huge part of the extreme slowness of the FDA is that many things that could easily be done in parallel are only done sequentially, and I bet every email between the wrangling lawyers took a full week to get sent, when realistically it could have all been hashed out in a day or two if everyone involved was locked in a conference room.
^ Yeah. I don't think I mentioned this point in my blog post, but should have. The hashing out could have been done long ago.
Deiseach's comment is well taken. I agree 100% -- the more risk information available to the consumer the better. What they should have done is gone back and modified the labels for the older drugs to update them with the new warning info. It seems it was the contrast between the lack of warning on the old drug and the warning on the new drug that the company took objection to more so than the warning itself, although they had a case against that too. Apparently it's hard to update labels, though (?).
I've read varying accounts online of whether the FDA uses rolling reviews but generally speaking they do not. So they don't start crunching numbers on the Phase III trial data until after the application is submitted. I suspect the factory inspections don't occur until after stuff is approved, also, although I don't know.
As another example of how things can be done in parallel, at some point after the initial phase III trial enrollment(s) they allow have doctors to start prescribing it. That is the idea of adaptive licensing, essentially which I summarize here :
https://moreisdifferent.substack.com/p/a-laundry-list-of-possible-fda-reforms
That's a very reasonable point in your case, but I don't think it's a reasonable criticism in my case. If other drugs in the class had this risk, but didn't have the warning, and a new drug is SPECIFICALLY designed to avoid this risk, it makes no sense at all to put the warning label on the old drug.
I see a role for the FDA, for all my bitterness I wouldn't necessarily burn it to the ground. But in this fight the pharma company was entirely in the right to insist on no warning label. "If they had stopped protesting something indefensible they could have helped you sooner" is true... but doesn't shift the blame.
What is the actual number of babies that died due to this? Definitely hundreds? (I saw a claim on Twitter it was more like “ten or twenty” than “hundreds or thousands”)
I also question that number. Not because any single life isn't precious, but because it speaks to the profitability of the treatment *and* about the difficulty in finding enough cases for good studies.
Any word on the number requiring liver transplants or otherwise suffering permanent damage? it could be that the larger number is the total number of babies affected rather than the number that died.
A really good question. No system is perfect and if the worst example of this kind of failure mode we can come up with is tens of people dying where we have a treatment that we can’t get over the hurdle of proof of efficacy, then we actually might conclude that we have a pretty good system.
So why were they so sure that "it isn't the lipids"? Because it seems a bit less certainty on that could have helped a lot. Was that a reasonable position? Would the establishment still have the same kind of position in the same kind of situation or has something changed since then?
I would guess it's because biochemically speaking biologically-derived fatty acids are pretty generic, it's just a question of how many carbons you have in the tails, and where exactly the double bonds lie. They're all metabolized in approximately the same way, so far as we know. I think for a long time people just assumed they were essentially all interchangeable, kind of like whether your gasoline contains heptane or octane doesn't matter that much.
The realization that this was not so -- e.g. that unnatural "trans" fatty acids were strangely dangerous, or that it actually makes a difference whether your first double bond is 6 or 3 carbons away from the end (the difference between omega-6 and omega-3 fats) -- seems only recently to have become part of the standard wisdom. I'm not sure anyone has any good mechanistic grasp on *why* this is true, also. The only obvious lesson here is the usual one: biochemistry is absurdly complex and not infrequently counter-intuitive.
That said, the omega-3 vs omega-6 differences that are in the news lately are subtle nutritional effects, and not a matter of immediate life or death.
Yes, although in this particular case (the infant parental nutrition), the assertion is that it *was* a matter of life and death. That is surprising to me even now -- I would never have suspected that soybean v. fish oil could mean the difference between lethal liver disease and health. I do not doubt empirical observation, but from a theoretical biochemical viewpoint that is seriously weird.
Perhaps it has something to do with neonate liver function, which differs in some important way from adult (or even child) liver function, in which case it's even less surprising that people were surprised by it, didn't expect it. The body of research on how 6-week-old livers work, and how that might differ from how 6-year-old or 60-year-old livers work, is probably amazingly scant.
> The body of research on how 6-week-old livers work, and how that might differ from how 6-year-old or 60-year-old livers work, is probably amazingly scant.
If the differences aren't known, it just raises the question again of why the overconfidence that it wasn't the lipids? Seems like extraordinary hubris to just assume neonatal liver function has no differences. You'd think we'd have learned better than to make assumptions without empirical data by now.
I don't think it's a question of hubris, it's a question of priorities. People are going to be very desperate to try to save a dying newborn, but they have to focus, because you can't try everything all at once, there just isn't time, or the resources. So they have to make some hard choices -- heck, this happens in medicine *all the time,* oncologists and surgeons and their patients make these choices every day -- do we try this or that first? Because once we make this choice, we're committed, there's no do over...
So they maybe say, gee, based on what we know, we think X or Y is more likely than the lipid content of the food. In this case, they turn out to be wrong. But that, too, happens all the time in medicine. Unless you were there, and know what X and Y actually were, and what the reason was for thinking they were more likely to be the issue, it's very hard to second-guess this and be confident they were idiots, as opposed to people who didn't have all the facts we have trying their best.
Even if I agree with everything you just said, the claim wasn't "we think lipids is less likely than X", it was quoted as "we know it's NOT the lipids". These are very different claims.
And even if they turn out to be *correct* that it's not the lipids, that doesn't rule some other compound present in fish oil that makes the difference, so ruling out the fish oil because of the lipids is also jumping to conclusions.
Reading the account in Scott's post they seem to have done tests with soya oil and without soya oil in such formulas, and similar problems happened in both, so they said "well it can't be the lipids since the same adverse results happen even without the soya".
If, for instance, the babies on non-soya formulations had done better, then they would have accepted "okay, the problem is with the soya oil". But of course, it was more complicated than that and not as simple as "replace the soya with fish oil".
I read a little more on this, and the suspicion is that it's a high fraction of polyunsaturated fatty acids, which actually starts to make sense, since each unsaturation is a hiccup in the normal process of beta oxidation of fats, and requires some extra hokey pokey step, and maybe baby livers don't have the latter ability fully developed. But this is just a wild guess.
it's unclear whether the "without soya oil" formulations had much oil at all - it's possible they were testing whether presence of Omega 6 was harmful and missing that the problem was absence of Omega 3; or, more likely, something more complex like the ratios
In the "relative balance" kind of problems, this is definitely true. In the rarer "actual full-blown deficiency, your body is not getting enough to meet requirements" kind of problems, it is not - deficiency of essential fatty acids (a.k.a. "protein poisoning" a.k.a. "rabbit starvation") is the only known form of malnutrition that can kill faster than simply eating nothing.
Curious about PNALD in Europe. This story is a little different than a new discovery of an alternate use for an older drug or something where there isn't already a body of evidence that it works. If Omegaven was already a standard of treatment there, was there a correspondingly lower incidence of PNALD? Might that have been valid data to use along the way?
That’s my thought as well!
Yea, for a complete outsider, this seemed weird to me aswell. I had the impression that science is an international endeavour - so wouldn’t studies on outcomes in Europe be valid? The USA babies could be a huge control group for European ones.
Or are confounding factors between continents too large (eg different quality hospitals)? Or is the number of affected babies still too low? Or do the European babies get other problems more often? Something else?
What is missing from the story in this regard? (Perhaps something obvious for someone with knowledge in medical sciences?)
Science is the same everywhere. But regulations are different.
Differing approvals in different jurisdictions is about regulations, not science.
Yes, I get that; but as I understood it the American regulators were apprehensive due to a lack of (good and large enough) studies. Given that different formulas were approved in the different continents - shouldn’t it be easy to write a study comparing the results (and use that to convince regulators)?
Since saying “babies in Europe in this situation survive 4x as often as in America” in a study seems so obvious, I am sure that there is something missing (in my understanding at least - but maybe also in the story).
Not medically related, and nearly forty years out of date, but when I was doing milk powder testing in a creamery co-op lab, it was for export and had to comply with US regulations, and those were more stringent than the at-the-time acceptable ones in Ireland/EU.
So certification and standards did/do differ from one country to the next, and simply saying "it's fine in European country X" doesn't mean it is legally acceptable in the US.
Just to be clear; I never intended to make the argument that regulations/certifications/standards are the same/similar, nor that “it’s fine there” is/should be enough. Moreso I wonder whether the liver disease (or other issues) were actually obviously gone where the alternative treatment were used.
Yes — I was suggesting the same. Not that the European regulations could apply in the US, but that the European market vs. the American market might effectively function as a giant study with a control for children who did and didn't receive Omegaven. It would still need to pass American compliance requirements but might provide a lot more data than the few US trials.
To put it much stronger: shouldn’t there have been a large and obvious difference in PNALD incidence? And wouldn’t that fact alone be evidence much stronger than that of any of the experiments?
Cowboy is still mostly a compliment at a certain hard-charging aerospace company, from my experience.
I'm about as cowboyish as it is possible to be and still be employable, so of course I thrived at a tech startup and hated working at a giant tech company. The increase in size of companies due to M&A or economies of scale or natural monopolies in tech is probably driving the culture away from cowboyishness.
In the least cowboyish culture I can think of, Japan, for a long time there were giant conglomerates dominating every industry, all headquartered within a few hundred miles of the God-Emperor's palace. In the wild west, there were tons of small businesses far from the reach of centralized authority.
Isn't everywhere in Japan within a few hundred miles of everywhere else, in particular the God-Emperor's palace? I guess you mean "the greater Tokyo area" in which case it makes sense.
I assure you it is extremely pejorative in finance (at least post 2008...)
It's real fukin bad in trades; cowboys are the clowns that get 95% of the way to saving an hour, and end up adding 3 onto the end of the job.
"Cowboy" is very bad in software engineering. At least my section of the field.
It was a derogatory term in Tombstone, AZ, and much of the Old West almost from the start. "Cowboy" basically meant "robber", with a side order of if they were the first settlers in the area and there was *nothing* to steal, they'd set loose some cattle and sit back until they could steal everything the cows had built.
Western novels, movies, etc, rehabilitated the term in roughly the way that "The Dukes of Hazzard" rehabilitated the Battle Flag of the Army of Northern Virginia. And we need *something* that symbolizes the general ideal of disrespecting established authority while still behaving honorably and engaging in productive enterprises on one's own terms. If there's cause for one or two of those terms to go away because of a preexisting and dishnorable usage, that's one thing - if they *keep* getting deprecated then it's going to look like someone is demanding submission to their authority and that all productive enterprises be done on their terms or not at all.
It sounds like your qualm is with how American healthcare law is written, not with how the laws are administered by the FDA.
As an example, who are you more mad at: judges who upheld disparate sentencing guidelines created during the war on drugs, or the legislators who wrote the laws?
Thank you.
My own take: the legislators are the most to blame. The trial judges are most likely often/usually blameless, but appellate judges arguably are not. Disparate sentencing guidelines arguably infringe on the eighth and fourteenth amendments, as well as some states' own constitutions.
Plot twist: some of the legislators who supported those harsher penalties for crack were black congressmen who saw the devastation crack caused in their communities in the 80s. So long as the demand curve slopes downward, harsher penalties = less crack. I don't agree with the drug war, but it's not such an easy mistake that I'd hold it against them.
And I honestly don't know if I could have done any better in their position, with the information I would have had at the time. Nevertheless, they're responsible for the consequences of their actions, including the unintended ones - to some degree that's the price you agree to when you choose to be in charge. Being wrong isn't something we should condemn anyone for, provided it's an honest mistake and people try to learn from them. Sadly this seems to be an unpopular position.
More importantly, all subsequent legislators are responsible for not having fixed the problems caused by previous laws passed.
> Disparate sentencing guidelines arguably infringe on the eighth and fourteenth amendments
Why? As far as I understand, the 14th Amendment forbids unequal treatment based on who you are, not what you do (such as what drug you sell).
What I don't understand about this debate: If powder cocaine and crack have mostly the same effects, but sentencing is much harsher for crack, then why doesn't everyone who sells/uses crack switch to powder cocaine? And are we sure that whatever difference makes people use crack doesn't also justify the harsher penalties (based on the assumption that drug prohibition is justified at all, which I don't think it is)?
crack is stronger and thus probably more dangerous than powder cocaine. It would make sense to have harsher penalties for dealing more dangerous substances.
IANAL, but my understanding is that if you can demonstrate racist intent on the part of lawmakers, then you may be able to claim the disparate impact is in fact a form of unequal treatment based on who you are. Hard to prove, though, just like it's hard to get any results arguing about unequal enforcement of laws against different groups of people.
Disparate impact is part of some anti-discrimination laws such as the Civil Rights Act of 1964. It's less clear to what extent it applies to constitutionality under the 14th Amendment. It looks like it may apply if discriminatory intent is proved, but it's hard to prove.
True. But FDA is also as dysfunctional as any major federal agency.
You can also take a step back and observe that FDA is created by legislation.
Thank you for this story of heroes and a system that can be so cruelly imperfect.
I'm glad you plan to write more on this topic in the future, because it's so important, and your analysis of it has been incredibly enlightening. Do you plan to write about the approval process in other countries as examples of ways the FDA can be improved? Or as cautionary tales?
Wait, so the only evidence for the effectiveness of fish oil in treating PNALD is a poorly powered study and a few miracle cure anecdotes? Miracle cure anecdotes are a dime a dozen. You can easily find thousands of them for everything from homeopathy to touching a statue of the Virgin Mary. That doesn't mean homeopathy obviously works or should be approved by the FDA.
"We were confident we were onto something and began to discuss our findings with others at BCH. The response was not what we expected. We were told “Everyone knows it's not the lipids because liver disease happened with them and without them.” Some individuals would even pull me aside to ask “Are the results real?”"
Usually when one scientist is convinced about the miraculous properties of their discovery while everyone else is skeptical, it's everyone else who's right, not the one scientist. I have no opinion on who's right in this case, but I wouldn't be surprised if fish oil turns out to not be very effective at all in treating PNALD.
Google “pnald fish oil”
"The parents demanded to be taken out of the control group, and then all their babies got better" seems like fairly strong evidence to me.
It's not like you have much need to worry about the placebo effect with babies.
No, it really isn't. Pick a quack therapy that you're sure doesn't work, and Google "(quack therapy name) + testimonials". Here's the first result I got for homeopathy: https://www.homeopathyschool.com/the-clinic/patienttestimonials/
This is why bleeding and cupping were standard of care for centuries. Hey, we gave it to Patient X and he got better! But Patient Y died! Yeah, well he had [other factor]/[was already at death's door]/[a physician who didn't do it right].
The reason we enforce the unnatural and cumbersome process of double-blind RCT is *because* historically speaking we know it is incredibly easy for human beings to detect "signal" in "noise" and rationalize it with some elegant theory.
> No, it really isn't. Pick a quack therapy that you're sure doesn't work, and Google "(quack therapy name) + testimonials"
No, what you're doing is fishing for an outcome in a sea of noisy data (like p-hacking). What the doctors did was an introduce a specific intervention to a cohort and observing an undeniably large signal. These aren't remotely the same.
A lot of undeniably large signals are also p hacking or are otherwise mistakes. Especially when you don’t know the doctor or how many specific interventions to a cohort they or others tried
P-hacking requires a large data set from which you fish out a signal that suggests an intervention. The exact opposite is happening here: the intervention is made and the signal is immediately seen. I frankly don't know why people can't see how different these scenarios are.
There are of course the possibility of confounders, but this also shows the weakness of the insistence on RCT for everything in evidence-based medicine. Some studies simply can't be done with controls which means some important life-saving interventions will never be approved. It's pure nonsense, and evidence-based medicine advocates need to take the stick out of their nethers until they figure out a better way to handle these scenarios.
There are a lot of people making interventions, and a lot of interventions being made, and a lot of ways for one to make an intervention and conclude it worked when it didn’t. As a result, and for many other reasons, many errant discoveries are made. The discoverers tend to believe exactly as you said there. Not all discoveries are like this, but one should look for more proof than just “person said it work”, even if it seems clear
I periodically suffer from a disease that doctors say is uncurable. But even time I get it, I spin around three times and say "abra cadabra Ave Marias!", and within weeks, my symptoms are gone! The effect size is huge. My friends can tell I'm better within seconds, without even talking to me.
What uncurable disease do I sometimes get? The common cold.
By the way, where did you or Alsadius read that the babies all got better? Scott's post only said that Charlie--aka a single person--made a miraculous recovery. It also said "Someone else in Hong Kong tried to do a randomized controlled trial, but ran into the same problem - halfway through, the parents figured out what was going on, demanded their babies be put in the Omegaven group, and nobody had the heart to say no to them."
...but it says nothing about the outcomes for those babies.
> By the way, where did you or Alsadius read that the babies all got better?
I did the obvious thing: googled "pnald fish oil" and looked at the resulting studies. To be clear, I wasn't claiming that the patients Scott was talking about all survived, merely that the study I found said that none of the 215 babies studied from 2004-2015 given fish oil had died:
https://pubmed.ncbi.nlm.nih.gov/26962059/
See also this review:
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3823519/
"Premkumar et al. (62) reported outcomes in 57 infants treated for PNALD with parenteral fish oil between 2007 and 2011. All infants were under the age of 6 mo at initiation of fish oil and most were born preterm (median gestational age of 28 wk). Cholestasis resolved with fish oil therapy in 82.5% of infants. Ten infants died during the study period, though none of the deaths were due to complications of fish oil therapy or liver failure. Nine of the 10 infants who died did not demonstrate resolution of cholestasis: 3 had end-stage liver disease prior to treatment with fish oil and 4 died after redirection of care due to poor prognosis from multiple comorbidities."
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3823519/
> we gave them hydroxychloroquine/remdesivir/(?)ivermectin and they got better! Obviously it works!
This is only the most recent and publicized version of something much more routine and more common
History (and present) is littered with doctors convinced that their patients prove their treatments work but they don’t in fact. It’s really weirdly common.
I think that’s fair about IVM, but HCQ has been pretty convincingly shown to not decrease mortality and if anything increase it IMO
https://www.nature.com/articles/s41467-021-22446-z https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7665543/ https://link.springer.com/article/10.1007/s00253-021-11094-4
And prophylactic use also failed - https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0244778
https://www.ncbi.nlm.nih.gov/pmc/articles/pmc7523161/
Well if you think you can do better than Cochrane and want to smash your head against a bunch of published trials with different interventions, timelines, structures, metrics, and endpoints, there are lots of published trials, many of which end with something like “we found no difference between soybean oil and mixed oils in liver failure outcomes” and many of which end like “we found significant reduction in liver failure outcomes”. I read a few and have no clue
This may all be true.
Still not a reason to make the drug illegal.
Time to remember Ignaz Semmelweis, who got convinced about the miraculous properties of washing ones hands between an autopsy and midwifery work. Didn't do him good. Of course, physicians are much more rational today, aren't we?
“this drug is too good so it would be unethical to have a control group”
Around 15 years ago, my wife was taking chemotherapy as part of the regimen to treat her colon cancer. Her oncologist asked if she'd be interested in participating in a trial of a new chemo drug. (I don't recall its name or maker, sorry.) She was interested... on the assumption that she'd be in the non-placebo group.
Not long before the trial actually began, she told me they'd decided to change it so that *everyone* would get the new drug. No one with cancer wants to be in the placebo groups!
Obviously not a double-blind study anymore. I'm not sure how the maker managed that switch or what exactly their new trial design was.
This usually happens when the drug is shown to be spectacularly effective and continuing administering placebo would thus be unethical.
Presumably the control group in such a study would be being given the current standard chemo-therapy; it would be insane for many reasons to give them no treatment at all
Yes, I agree. And I think that was the case, in fact.
'Placebo' was a too-hasty & poor description, IOW.
> It will be really hard to change the system radically enough to make it possible to do better, but I still think we should try.
Lots of people agree with this, but the political coalitions in the US make this impossible.
If you read Charles Murray’s By the People and Krugman’s Conscience of Liberal (both MIT PhDs), they are actually 100% on the same page about who the players are and how the coalitions arose.
According to Murray, he believes the post 1950 government is illegitimate, and the last time America was still America was the 30s. And that people who think like him took over the Republican party starting with Barry Goldwater, via a few influential publications and right wing think tanks funded by a handful of wealthy families (who he thanks individually, and where he has worked most of his career).
Krugman is exactly on the same page, with the people, institutions, ideology, and timeline.
Krugman’s main counter-point is that the broad middle class that emerged after WWII was not the natural outcome of technology but massive interference in the market, and the natural market outcome is industrial-revolution like inequality. Which I am also convinced is correct for reasons too long to get into here.
An ideal state would combine the polices that maintain a less unequal distribution of income (which is probably needed to maintain political stability in democratic countries), with heavy doses of capitalism and markets. There are certainly people in the center-left that support this. But they are in a coalition with the left, because after the Republican party takeover not enough common ground exists to make policies that keep inequality in check.
So, maybe we need radical change. But with these coalitions, what radical change is possible? Where is the radical change going to come from? Until we can answer those questions adequately, we should be careful what we wish for.
Thanks, I haven't read it, I'll take a look at it. The thesis seems plausible for much of history, although it's hard for me to imagine that policy instruments are useless in modern states that have a much bigger economic roles than in the past.
> In any case, economic inequality seems to me a rather dilatory pre-occupation in itself
Regarding inequality there are three major issues that come to mind:
Is it a bad thing?
Is high inequality the natural state?
Why did it decrease after WWII?
To briefly address the first point, it's obvious that equality is not a good on its own - you could have an equal society with everyone poor. The reasonable concern with inequality is if the consequences interfere with having a society that is largely not poor. I think the better description of the ideal is "broad middle class" - what matters is the average standard of living. There could also be secondary sociological / political concerns that Charles Murray talks about in "Coming Apart"; in addition to the economics people are increasingly living different lifestyles and so can relate to each other less well. But I'll leave that topic aside here.
For if it conflicts with the middle class - the economy is not a zero sum game but there are resources that are scarce, such as human labor. People who are employed by high consumption by high income earners are necessarily not available to work on other things.
To go a bit deeper, my favorite lens is sectoral balances. Unless you print dollars, there is a limited amount of money in the system. A $100 loan is a $100 asset on one side and a $100 debt on the other, which adds up to zero. This creates certain simple constraints in thinking about what is possible that don't require any advanced math.
For example, you can divide dollar balances into foreign / domestic government / domestic private sector. It all has to add up to zero. If you have a trade deficit, that means the foreign sector has a dollar surplus, and the combination of the domestic government / private sector must being going into debt.
Similarly, big tech companies with imperfect competition are accumulating big cash balances. Who goes into debt to sustain this? Can the debt be increased perpetually? What happens when it stops? Another factor is the "paradox of thrift" - people may want to save dollars at the same time, but that's mathematically impossible, it has to add up to zero. One person's savings is an other's income, and by trying to save both fall and everyone produces less.
I'll stop there, Ray Dalio has some good stuff if you want to look further. But I think if you analyze things in this way enough it suggests the laissez-faire approach has some problems and thinking things will just work out remotely well on their own is probably naive.
I'd like to add that if things *don't* work out, there are political implications, and there are no guarantees that laissez-faire or many other political things can be maintained. The choice is not straightforwardly between the welfare state and libertarianism.
I'll skip the second issue (is it the natural state) since it seems like you might agree there already.
For the last question, "Why did it decrease after WWII?", Krugman argues tax rates, labor unions, and cultural norms (for example, the unions might object to executives paying themselves $100M, and there was less reason to do that anyway due to the high tax rates).
It's obviously a complex topic, and this wasn't the answer I wanted to hear (I still have some reservations about unions). But sorting through the arguments (and applying additional economic thought experiments) I've become grudgingly convinced that there's a case for policies that are a bit more left-leaning than I was originally inclined to, and also that maintaining freedom, efficiency, and stability over the long term is a very tricky business.
"Another factor is the "paradox of thrift" - people may want to save dollars at the same time, but that's mathematically impossible, it has to add up to zero. One person's savings is an other's income, and by trying to save both fall and everyone produces less."
(1) If you are putting your money in the bank as savings, then that money goes out in the form of loans so it is indeed being spent by others as a form of income and goes back into circulation in the economy
(2) If you are stuffing your money under the mattress, it is taken out of circulation. It is not being spent as income, true; production is less, true; but one day your heirs will find this windfall and spend it themselves.
On (1), if you put the money in the bank, and someone else takes out a loan, that doesn't describe "people wanting to save dollars at the same time" (at least how I meant it). One person is saving and another person is going into debt, and it still adds up to zero.
But in many circumstances more people may want to save than go into debt (for example, if the future economy looks less rosy), and this is where the problem arises. What happened after the financial crisis was the government went into debt, which enabled the saving by the private sector. There's a tendency for people to feel saving is good and debt is bad because that's true on an individual level, but on the macro level they are the same thing - two sides of the same coin. You can't just have everybody save without printing money.
On (2) this only works on an individual level and not a macro level. For example you can't have everybody save up money for 20 years, then have the whole economy retire.
Money is basically a promise for future goods and services. If that person stuffs their money in their mattress, you don't actually *want* production to be less. The important thing on a macro level about being able to fulfill that promise is the ability to produce *future* goods and services. But producing one less car in 2000 doesn't help you produce a car in 2020. Most goods and all services can't be stockpiled. And drastically cutting back on current production may even harm your capacity for future production.
This is why you see some emphasis on infrastructure - of the available things to spend money on, it's one that has some plausible positive effect on future ability to produce goods and services (for example, by people spending less time in traffic).
To tie it back to inequality a bit: wealthy people save a greater percent of their income. On the other side of that it drives the interest rate down, which increases the price of housing and the amount of debt people go into and that balances out the saving. But now interest rates are about 0, and can you easily go lower than that (and keep the debt going up to balance the saving)? And if people want to save but not go into debt, you don't end up saving but just produce less...
My understanding is that Murray endorsed a guaranteed minimum income.
Yes, later in life he endorsed it. I myself think Milton Friedman - style libertarianism with UBI is a great compromise between enabling a middle class and libertarian values, it's a simple plan to address the problem without too many adverse incentives, while maintaining small-government elsewhere. It's an extension of the capitalism-with-safety-nets that most successful countries seem to follow, just much more to the libertarian side.
His party isn't with him on that though, and doesn't look like it's heading in that direction. So I think it's not particularly relevant to the political coalitions as they stand.
He endorsed it in "In Our Hands" in 1997 and "In Our Hands" in 2006, whereas "By the People" was in 2015. Part of the reason he endorses just giving people money is that they would then have the freedom to do whatever they want with it... and his complaint in "By the People" is that regulation is inhibiting such freedom.
My point was about changes to the Republican party starting in the 1960s. Murray's UBI recommendation is something that happened later, and the Republican party hasn't adopted it.
I think what you are talking about, UBI and major deregulation (Milton Friedman was against professional licensing for example), is a compelling idea. There would certainly be advantages and disadvantages, but I think it would likely be better than the current system (although personally I might want to run a small-scale experiment before scaling it up). If the Republican party focused on that instead of, for example, corporate tax cuts, I think they might be more popular overall and it would be more beneficial to the public. Unfortunately I don't see any reason to believe things are heading in that direction.
I don't think US corporate taxes are that low by international standards. European welfare states are mostly funded by consumption taxes like the VAT.
Uh, I didn't say they were? Previously the nominal rate was high but the effective rate was substantially lower than the nominal rate. And it looks like VAT makes up on average 20% of tax revenue in the OECD, behind income tax and social security contributions (as of 2018).
Look, it's not like I've never seen a Republican policy I liked. Immediately before the rate cut bill, they were attempting comprehensive corporate tax reform (with a border adjustment tax) which I thought was a clever idea that addresses the tax haven issue, without needing any international cooperation.
But even just in terms of pushing libertarian principles, I think it would be better for them to prioritize middle class tax cuts and reducing costly regulations (and they do do *some* of this). Both because that's more in line with academic economic thinking, and from a basic intuitive standpoint that corporate earnings and the incomes of the wealthy were already at record highs, so borrowing a lot of money to increase their after-tax incomes further didn't seem urgent.
As it is the establishment is losing credibility with the base, and they more or less didn't even campaign on the tax bill even though it was the main piece of legislation they passed. And to the degree that the new faction that is replacing them has a coherent ideology, it doesn't seem to be particularly libertarian.
It doesn't take 10 years to hear back from the FDA after filing Scott. PDUFA dates are within 10 months of filing an NDA
There's a big difference between the published timelines and the real timelines. I only have to deal with the FAA, not the FDA, but it's safe to say that the problems are endemic.
Nobody every got fired at the FDA because a PDUFA was responded to a few years later than the published guideline.
Why is not having a control group considered such a deal breaker? I know it makes it hard to tell if there was a placebo effect, but if an experimental treatment is much more effective than a placebo possibly could be, shouldn't that be enough?
Good practice, anal-retentiveness, and/or fear of change.
The issue isn't so much about placebos as it is about selection bias. When you do the study, some percent of the patients of the patients will recover according to your success criteria. What do you compare that percentage to in order to determine if the drug is effective?
If you compare to global or nation wide results, there is the potential confounder that not all hospitals are equal. If you compare to results at the same hospital at different times, there is the confounder that medical practice changes over time.
Whoever you compare to is your de-facto control group. If they aren't randomly selected from the same pool of patients, there is a significant selection bias associated with who is participating in the study. This is all before you consider the question of placebo effects.
If the drug is so good that almost everyone who doesn't get it dies, and almost everyone who does get it lives, that may be good enough without the explicit control group. But your evidence will almost certainly be weaker than you expect. Intuitions are misleading in such cases.
Ah, that makes sense! Still, if there's no obvious selection bias and the results are promising enough, they should approve it.
I definitely agree, particularly in this case. I just want to separate the scientific question of how to evaluate the strength of our evidence from the moral question of how much evidence is necessary before a drug should be permissible to proscribe. I think the FDA's failure is largely on the latter question rather than the former.
I think I missed something. In Europe they use fish oil based nutrition. So wouldn’t rates of PNALD be much lower in Europe? Is the advance that it has to be pure fish oil?
Since I can't like things here, I'll just say that I had the same thought - you'd think there'd be sufficient European data to lean on.
Retrospective studies are always difficult and rarely definitive, simply because it's extremely difficult to match your experimental and control groups. It's unusual for the required demographic and medical information to be fully present, and to be sufficiently confident that everything *other than* your experimental variable was held constant. Remember this is medicine, and moreover a rare condition, so we are inherently in the realm of very small and very noisy data sets.
Amazing. This ultra obsession with randomized controlled trials is crazy -- complete brain dead. Come on I get it. If you want to pick up very small effects yes you need them, but if your expectation are big results just use some bloody common sense. You will be able to see them if they do exist. The base line is that all babies die. If you save 20/100 you have your answer in front of your eyes.
The article doesn't have the numbers, but it sounds like it's more like 1/100 babies die with the soybean oil, and the fish oil saves that 1/100.
"Lots of people on parenteral nutrition get PNALD. Some need liver transplants. Others die."
If that's the rate, you need to have a sample of hundreds of babies with digestive tract issues to have statistical significance.
If that is the case you are right.
> If that's the rate, you need to have a sample of hundreds of babies with digestive tract issues to have statistical significance.
Fair, but follow-up studies showed *none* of those 1/100 babies died in the timeframe from 2004 to 2015. I think the signal is pretty clear even without an RCT.
The use of double blind studies of sufficient sample size is critical. But for many kinds of large scale research, the system is breaking. Mostly because the hurdles to get through multiple studies of incrementally larger sizes, and different designs, and different standards of proof. All the while trying to weigh risks and benefits for the control and intervention group.
I'm most familiar with how to evaluate genetic effects and risk factors in large populations, This is maybe a good example of how it works in medical interventions.
There's a better, more analytically principled, and powerful approach (we use this for optimizing game settings and ad placements in my current job). It's to use (usually Bayesian) models of likelihood to probabilistically assign treatment. Like, if there is a high risk, maybe-good new formula, you'd assign it at very low probability. If it has great results (the likelihood it's better goes up) and you start prescribing it more. As the sample size goes up, but the results begin looking worse, you assign it less.
Importantly, you don't need to give it to thousands of people off the bat and risk so many lives. But you can ramp it up quickly as it becomes more obvious that not doing so risks lives.
That does sound like a great solution! It won't work for everything since there have to be people willing to try it though ("here's this drug that might cure the common cold or might kill you"), but it seems like it would've at least been useful here.
The problem is you're assuming in this scenario that the question "is it working?" is easily answered. And that may well be the case in many fields, but it is most certainly not the case in medicine. There are a host of reasons, from the psychological (in both patient *and* physician/researcher) to the statistical and even definitional ("what do we actually precisely mean by 'working?') that make this a major challenge.
One of the most important reasons for doing double-blind prospective studies is to prevent these factors from having too much influence, and in particular preventing the inevitable human psychology effects in situations of enormous emotional weight from screwing up the results. People have *a lot* on the line in these trials: the patients are often betting their lives or health, the physicans are betting their self-respect and peace of mind, the company may easily be betting billions of dollars or their entire corporate existence. If the trial was not blinded, as you suggest, and things were continually evaluated openly, it would be very difficult to prevent these psychological factors from playing a big role.
I mean, after all, this entire post stream grew out of the feeling that the aducanumab approval was driven by too much emotional and/or financial investment in a positive outcome.
Reread my first paragraph. The issue isn't the double-blinding. It's the need to start with a "big enough" sample size, get all the data at once, and do several full rounds of the analysis-different proposal-grant-experiment loop.
By the time you've gone through this a lot, it's possible to lose steam, or fail to get funding. Or see a few null results. And this can all be hobbled by experiments that use very weird baselines, null hypotheses or just engage in p-hacking.
Updating Bayesian priors on the fly, with transparent likelihood models, and risk-benefit correction is a lot more tractable, and effective.
Yeah, but see I'm disagreeing with you. I'm saying *without* blinding, you are basically going to generate noise, and not any useful signal, because of human nature, and any effort to optimize the efficiency of generating noise is ipso facto wasted.
You might be disagreeing, but you're not disagreeing with me. I'm saying "blinding and large sample sizes are important."
Reread what I wrote. I said nothing about unblinding samples. Not a thing. In fact, I emphasised *twice* that they're essential.
OK, then I'm disagreeing with the opening sentence of your 3rd paragraph, which begins "There's a better, more analytically principled, and powerful approach..." I don't think the approach you describe is better, because I don't think it would work, because blinding/human nature. Sorry for the lack of clarity.
You can still use a blinded design with the approach I described. You SHOULD use a blinded design with the approach I described. Blinding good. Sample size good.
bad: "Mostly because the hurdles to get through multiple studies of incrementally larger sizes, and different designs, and different standards of proof. All the while trying to weigh risks and benefits for the control and intervention group"
I'm fascinated. If you read a restaurant review that praises the chicken and condemns the fish, do you say "wow that guy really hated the chicken"?
Aducananumab would be a good example of how you would do the Bayesian approach, and why an artificial threshold like "FDA Approval" is totally broken.
You can start by offering the drug to some people, and sure, as there are few bad side effects at first (it's not gonna kill you) and it's clearing up some plaques, you'd ramp up the amount on offer.
But over time, you're going to see (again, sure, it's not going to kill you) and maybe okay, you can approve it, but the marginal benefits on long term cognition are basically 0 (it's as good as the control). A simple likelihood model, with a cost factor, would quickly tell you not to prescribe it.
We probably would have started ramping up vaccine distribution 6 months earlier, for the mRNA vaccines, under a similar dynamic testing regime.
You have to have some sympathy for the FDA as they're being simultaneously criticised for not bending the rules because BABIES WERE DYING and for bending the rules because GRANNY HAS ALZHEIMER'S.
Don't respond to heartstrings-tugging: heartless paper-shufflers
Do respond to heartstrings-tugging: brainless jellyfish
The objections are because Aduhelm hasn't any evidence that it does any good for Alzheimer's but for the fish oil there also wasn't any evidence that it did any good (the warning about efficacy are still up on sites about it):
Limitations Of Use
Omegaven is not indicated for the prevention of PNAC. It has not been demonstrated that Omegaven prevents PNAC in parenteral nutrition (PN)-dependent patients [see Clinical Studies].
It has not been demonstrated that the clinical outcomes observed in patients treated with Omegaven are a result of the omega-6:omega-3 fatty acid ratio of the product [see Clinical Studies].
I very much want to hear about what the default-yes and default-no-but-with-smaller-hurdles models look like to you! *refreshes inbox excitedly for those posts *
FYI, Eliezer wrote about that story in 2017.
https://www.lesswrong.com/posts/x5ASTMPKPowLKpLpZ/moloch-s-toolbox-1-2
Man, the timeline on this patent tells a story. https://patents.google.com/patent/US20060127491A1/en
I feel like it may be worth noting as a small aside here that soybean oil consumption has increased significantly in the US in the last several decades and is put in, well, just about everything I look at here. Meanwhile, fish oil is not only much less common in our standard diet, but so uncommon that it is taken as a dietary supplement by millions so that they get a small amount instead of almost zero.
I think it's likely the case that at least one, if not more, of the following is true:
1) excessive soybean oil (and similar oils, perhaps high in linoleic acid) consumption is very common and very bad for you
2) same as the above, except that it is primary bad for you when used at high temperatures such as for frying
3) the ratio of omega 3 to omega 6 fatty acids in our diet is terrible (with most of us having far too much omega 6 in comparison), partly as a consequence of the large prevalence of soybean oil and large lack of omega 3 oils such as many fish oils, and this is probably very bad for us. This may also be why fish oil supplementation seems to help some groups of people while doing nothing for other groups of people
I'm still only moderately confident in some of these, but they do seem to be relatively convincing propositions the more evidence I find in their favor, and soybean oil is probably not the best oil we could be giving to an infant regardless here, so I kind of frowned when I read that it was the standard, when it is also the first ingredient in a lot of pretty terrible things. Conveniently, it is also extremely cheap, so perhaps I should not be surprised.
It seems like the new recipes for neonatal lipid formula is are mixed and include soy and fish oil. So I’m not sure that it’s a case of “soy oil bad” as much as “soy only is bad”
Also, for the natural health perspective, I think many peoples of the far past lived inland and were quite primitive hunter gathering on lots of natural animal and plant fats and no fish at all.
That said I don’t eat isolated plant oils at all and do lots of fish and animal due to more general suspicions of modern food, but the whole oil composition thing has always seemed like the evidence isn’t strong enough either way to tell.
In various farmed fish mixed soy:fish oil preparations are much better for growth than either soy or fish oil alone, and 100% soy also increased liver fat. I was looking for studies on people, but thanks google!
A combination is probably a decent bet, I intended to convey that point with possibility number 3 in my comment; that it may be the case that soybean oil is not harmful unto itself, but rather that an overdone ratio of its constituents may be.
As usual it's very difficult to have high confidence on these propositions when most of the good studies we have are not in humans (or don't even exist to begin with!) and the data we have in humans is generally terrible and only correlational, but I do think it's a nice area to at least spend some time looking in, agreed.
Yeah, unhappily it's not until *after* people start keeling over dead that anyone is motivated to find out "oh crap, soya oil in high quantities is not good for you".
It's a lot easier to take it as read that soya is safe, because look at all the countries that have been eating lots of soya and soya products for centuries and they're not keeling over dead.
Indeed not, the Japanese eat a metric crapton of soy every year, and they have among the highest life expectancies and lowest rates of heart disease in the world. If memory serves they *do* however have higher rates of liver cancer than we do...
Don't the Japanese also eat lots of fish oil every year, though? My understanding is that Japan is a massive outlier among developed countries in how much fish is commonly eaten.
Yup. God damn it, messing up the experiment that way. We need a God-Emperor who would command each country to differ in its diet from other countries in only *one* particular...
An explanation I've heard is that whole soy foods (edamame, etc) are safe, but processed soy foods aren't. Same goes for things like canola oil, the actual oil isn't so bad but the processing to extract is so harsh that it becomes unhealthy.
If there is a harmful effect from excessive O6, it is likely some longer term (over decades) increase in various types of cancers (via systemic inflammation?), and perhaps subtle mood disorders. But there are few control groups since practically everyone uses "vegetable oil" nowadays.
I've nearly eliminated excess O6 from vegetable/seed oils in my diet, it was pretty hard actually, it is used in a LOT of foods as a fat substitute, and is ubiquitous in restaurant food. Fresh bread is one product where you can easily tell the difference between oiled and non-oiled, oiled is "harder" especially when toasted.
The thing I'd heard was that fermented soy products like tofu are good, but just plain extraction like soy flour and soy oil aren't.
Like basically all health diets, the benefit probably isn’t the O6 but the fact that you stopped eating restaurant foods and the sorts of foods with fat substitutes. That’s my guess for what the real benefit is. Idk
I'm generally skeptical of arguments that ordinary food processing processes are making any chemical changes to food constituents, since these generally require harsher conditions.
Most of the time I suspect any benefit of less processing inheres in avoiding changes in the mix of nutrients, which among other thing scan interact in interesting ways with your own gut microbiome, or possibly with changes in the microbiome that lives on the food itself, e.g. I might hazard a guess that fresh carrots plucked from your own backyard garden and rinsed under the tap are better for you than factory farm carrots power-washed in a giant machine because those from your backyard still have a variety of healthy soil bacteria on them.
But that could just be because I'm biased to think the biology matters more than the chemistry per se, and/or because I have a vague suspicion the influence of our bacterial ecosystem (in our guts) is way larger than we have thought -- which may help explain otherwise baffling aspects of the nutrition/health matrix: it matters not just what we feed *our* cells but what we feed the billions of bacteria in our guts, and how that changes their relative demographics.
Re the hunter-gatherer diet. One traditional way for some coastal indigenous cultures in the US was drying or smoking fish, making fish jerky. Then it lasts a long time and can be traded. Also, inland rivers and lakes used to have many more fish than they do now; I don’t know about relying on fish but accessing fish should not have been difficult even for non-coastal groups.
I have theories about regional human adaptations to high mineral content in water sources. I don’t know of any studies on it though.
Actually you’re right about rivers, whoops.
I just meant that a decent proportion of primitive peoples didn’t eat any fish, and skimming a few papers seems to confirm that (and many ate some fish and many eat a lot of it... the only real constant in the behavior of ancient/primitive peoples is how much it varies, and humans seem adapted to that)
Digestion is complicated! It makes sense - I was googling just now, apparently there are 3 subtypes of omega 3 with plants (nuts/seeds, algae, a few other things but not soy) being a viable source of at least 2.
It could have come down to something as simple as the USA has a lot of soya crop production while Europe doesn't, so when making up the formulations the German company went for "well we can easily get fish oil".
And if I go by the warnings, it's not uncomplicated story of "fish oil good, soya bad" because Omegaven has adverse side-effects as well:
https://www.rxlist.com/omegaven-drug.htm#warnings
Though soya oil does seem to be more risky:
"Risk Of Death In Preterm Infants Due To Pulmonary Lipid Accumulation
Deaths in preterm infants after infusion of soybean oil-based intravenous lipid emulsions have been reported in medical literature. Autopsy findings in these preterm infants included intravascular lipid accumulation in the lungs. The risk of pulmonary lipid accumulation with Omegaven is unknown.
Preterm and small-for-gestational-age infants have poor clearance of intravenous lipid emulsion and increased free fatty acid plasma levels following lipid emulsion infusion. This risk due to poor lipid clearance should be considered when administering intravenous lipid emulsions.
Monitor patients receiving Omegaven for signs and symptoms of pleural or pericardial effusion."
At the same time, other websites are claiming soya oil is good for you, though they do point out the Omega-6 imbalance:
https://www.healthline.com/nutrition/soybean-oil
https://www.healthline.com/nutrition/soybean-oil#Potential-downsides-
Fish oils aren't the only source of omega 3 fatty acids, they're just unusually concentrated sources and also provide the most-readily-usable omega-3 fatty acids. There are plant sources, like some seeds and nuts. Even beef, chicken, and eggs would have traditionally been good sources, but the amount of omega-3 fats in them depends on the diets of the livestock, so grain-fed livestock have much less than pastured animals.
"One small weak randomized trial wasn’t enough evidence for anyone else to care. But it was enough evidence that BCH refused to do larger studies, because that would require putting some babies in the control group, and obviously those babies were going to die, and that would be unethical."
Wonder how many drugs are stuck in this dilemma.
Not just drugs. It took a long, long time for the "lumpectomy" approach to treating breast cancer (as opposed to radical mastectomy, taking everything) to become standard of care. Because who the hell wants to be in that experimental group? "You've got a cancer. This has always been treated by removing the entire breast, but we want to try just removing the tumor and a little bit around the margins. It might be just as good as taking the whole breast! We'll look at whether those in the control group live longer than you. Want to sign up?" Er...yikes.
It's a heart-warming story and I am very glad that now there is a proper feeding regime for babies like this.
But the bit about deliberately putting pressure on the pharma company by feeding negative PR to the media? That's how you get Aduhelm and drugs like it approved.
And the praise of cowboys - the reason "cowboy" became a term of opprobrium is because people have experiences of such types. Authoritarian surgeons who think they are 'saving lives' have caused suffering - there's one notorious scandal from the late 90s in Irish medical history:
https://www.irishexaminer.com/news/arid-30348600.html
So when it works out, great! But many times it doesn't work out.
Hey how about this cowboy?
https://en.wikipedia.org/wiki/Ant%C3%B3nio_Egas_Moniz
He got a freaking Nobel Prize for his brilliant pushing of the envelope work...er, until it turned out to be hideously wrong.
Wouldn’t a better way forward be to grant medicines approved in reputable places like the EU, UK, Japan or Australia an automatic provisional authorization?
That's been floated many times before – and it IS a good idea. Alex Tabarrok is the person I'm most familiar with who's brought it up 'recently', and repeatedly.
Sadly, it probably just doesn't work from a 'public choice' perspective, e.g. the first bad 'approval via proxy' would probably see it immediately reversed.
That Dr. Folkman sounds like a real mensch
His father was a rabbi in Columbus Ohio in the 60s and 70s at my grandmother’s temple! I think Dr Folkman, famous in the Boston area, actually got caught up in some scandals about his research, but I don’t remember how it all got resolved.
Not sure why the post says it takes 10 years to clear the FDA. Omegaven was actually approved in under 8 months. https://www.accessdata.fda.gov/drugsatfda_docs/nda/2018/210589Orig1s000Approv.pdf
And in 2018, the "median review time was 10.1 months for
standard applications (n=79) and 7.6 months for priority applications (n=43)" https://jamanetwork.com/journals/jama/article-abstract/2758605
Even if you count from the first study in humans to the ultimate clinical trial(s), total time is 9.1 years.
But that isn't because of the FDA per se, it's because biology is really hard, and we don't want to kill people. It isn't trivial to introduce a drug to humans for the first time, as the TGN1412 study tragically showed: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2964774/
Here's an easy way to minimize the risk of initial human trials that would never, ever, ever be possible, even though it clearly makes all kinds of sense: give it to terminally ill patients first, offering significant amount of money to them and/or their heirs. Of course, terminal illness is a huge confounder in the big picture, but it's a great way to at least make sure that the drug won't necessarily set one's lungs on fire.
What makes you think it's easy? It's already difficult to enroll enough very sick people in clinical trials for a therapy that might actually help them, I mean, where the payoff matrix at least includes one square with a positive number. You're going to try to enroll people in clinical trials where *none* of the squares have a positive number? Tricky.
And the immediate problem with offering large cash awards is that your enrollees will quite naturally be disproportionately poor and minority patients, most likely those without health insurance, people on Medicare/Medicaid, et cetera. Let us imagine your proposal passes and we find that these kinds of medical experiments, with the occasional gruesome results, are predominantly being performed on poor and minority patients. Let us imagine the political campaign that immediately follows the sensational news stories...
I mean, basically it's the same category as offering $1 million in cash to the family of any healthy young person who signs up to be a heart transplant donor, in order to solve the shortage of heart donors. You can imagine what the political response would be to such a proposal.
Yeah, I think it’s ridiculous anyone thinks that’s a reasonable objection. Of course people with less are more incentivized. That’s true everywhere and all the time at every level of wealth. It doesn’t mean poor people are cows without moral agency.
Hmm...well, OK, but it's pretty clear the rest of the world violently disagrees with you. You cannot sell yourself in to slavery, or answer Armin Meiwes's ads. If Union Carbide decides to cut corners in its Bhopal plant, but not the plant in Missouri, because the much poorer people in Bhopal (1) don't object as strenuously, on account of the good jobs at the plant, and (2) when a lot of them are killed in the terrible accident they settle for much less than the Missourians would, the world condemns Union Carbide as immoral bastards, rather than bidding up their stock because of their clever business strategy.
> I realize the hurdle is there for a reason. What would a default-yes medical world look like? What would a still-default-no-but-the-hurdle-is-smaller medical world look like? I have thoughts about this which I hope to address in future posts.
Given the history behind the term "snake oil salesman" and Coca(aine)-Cola, my presumption is that there used to be such a world. What were the events that led to the reversal? What lies behind Chesterton's Fence ought to be historical record, at least in part, right?
The historical record definitely shows the accretion disk - it has built up over the last 100+ years.
The FDA has history of itself at https://www.fda.gov/about-fda/fda-history/history-drug-regulation. Axes of safety and effectiveness were put in place by the early 1960s (due to sulfanilamide, thalidomide, Nazi experiments, others).
But it seems that today "low effectiveness plus high safety" is rejected. That category scoops up "not enough proof of effectiveness yet plus high safety" and so it's the efficacy hurdle that might cause the most problems of the type Scott described regarding Omegaven. "High effectiveness and high safety" can take a long time. "High effectiveness and low safety," I am ok with rules around that type of thing, although what type of rules is debatable. But I also think it is ok to put gates in Chesterton's Fence (once one understands why it was built), partly because in this case, the edifice really did accrete, it was not dropped on us in complete form but evolved over the course of a century in response to competing pressures. Those pressures aren't over, so it can keep evolving.
The European regulatory body the EMA functions differently. Here they are compared, "The FDA requires valid scientific evidence that devices are both safe and effective. By contrast, the European CE mark requires only proof of safety and that the device performs in a manner consistent with the manufacturer's intended use." (Retina Today, Sept. 2012, Shapiro & Lafond). So my impression for a while has been that more things get through in Europe. Like Deiseach said elsewhere they might not have specifically rejected soy in Europe, the fish oil formulation might just have been the locally developed thing which then turned out to be more effective than the US one in those situations.
Even the FDA sees that it may need to evolve its effectiveness axis:
""The randomized clinical trial is excellent methodology if you want to understand, on average, whether one treatment is better than another treatment," notes John Bridges, assistant professor at Johns Hopkins School of Public Health, "but if we think about a distribution of outcomes, no single person in the health care system is the average."" (FDA.gov)
> But it seems that today "low effectiveness plus high safety" is rejected.
I wish there was more rejection of that. As long as you have high safety, low efficacy therapies, doctors will gravitate to them due to sheer risk aversion. As a patient, I 'd rather take more risk for higher efficacy (assuming the disease is serious in any way)!
I clicked Scott’s link to the SSC post on the research study he had been doing that was swamped by the IRB. Very interesting. Here is an article that concluded what you were suspecting.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3830717/
Cerimele, Chwastiak, Dodson & Katon, 2013, “The prevalence of bipolar disorder in primary care patients with depression or other psychiatric complaints: a systematic review.” Psychosomatics, 54(6).
Screener - 20.9-30.8% prevalence. Full diagnostic interview- 3.4-9%. Whee.
Have not fully read this but I guess primary care has the advantage of repeat patient contact and permanent staff.
I'm really straining my memory here, but I think that the screening instruments were designed for a primary care environment but getting used in psych wards. CCD&K proved that they didn't even work for the primary care environment they were designed for. I wanted to show that they didn't work in psych wards either.
This is a really excellent post, and I commend your willingness to dig deeper into the story and add a thorough and thoughtful correction to your original story. It was instructive to read this, not only in terms of learning what actually happened, but in seeing how the story was oversimplified and in some cases distorted over the years and among a wider audience -- which is alas all too common.
Indeed, I think one of the obvious conclusions to be drawn here is that when it seems like some large group of pretty smart people has done something just so incredibly brain-dead that it only takes a moment's thought to realize how they went wrong, and how it could have been done better, that some uncomfortably high percentage of the time that conclusion is flawed -- that there is much more to the story than was at first apparent.
Of course, sometimes people, even very smart and experienced people, do make stupid or corrupt or evil decisions, so we can't rule it out all the time. Still...if I had to make a choice, I'd say the modern era could usefully move its collective needle towards patience and away from quick judgment.
The usefulness of the FDA has been studied a lot, and every study I've read has said it kills a lot more people than it saves, but gets away with it because it kills individuals by banning drugs that will save them, which doesn't seem as heartless according to American biases.
Well, as a dyed-in-the-wool empiricist, I am automatically deeply skeptical of any study that relies on counterfactuals -- evaluations of what the world *would* be like if the world we live in were radically altered in some way. So this kind of evidence is pretty weak by my standards. Doesn't mean I wouldn't be persuaded, if there were enough of it.
Part of the obvious problem, to my mind, is that predicting what the world would be like if the FDA behaved quite differently is very challenging, since it's so different from the world in which we actually live, and more importantly there are always dynamic effects, meaning you can't *just* say "What if the FDA approved more drugs with lower levels of evidence *and nothing else changes?" Other things *always* change, because people respond dynamically to changes.
Id est, if the FDA starts making it easier for drugs to be approved, drug companies will respond to that...in some way. Physicians and patients will respond...in some way. If the approvals are good, people will respond in some way, and if there are more that are bad, people will respond in some way.
Doesn't mean I don't think changes should be contemplated, by my preference would be to do things experimentally and incrementally and see what happens, rather than make any big change, especially one driven by hypotheticals. I think there have already been efforts along these lines, e.g. the "fast-track" approval protocol that got implement in the 80s when people were enraged about how long it was taking AIDS drugs to get approved, the "compassionate use" protocol that allows desperate people to try stuff, the "emergency use authorization" that allows them to signal some kind of pro-tem-we-might-change-our-minds approval that takes some of the pressure off to decide once and for all.
What I'd like to see is (1) evaluation of these efforts, to see if they are working as intended and are free of malignant unexpected side effects, and (2) creative thought in terms of what additional initiatives we can imagine that would experiment with different ways to approve drugs, so we can inch our way towards something that works better, but without taking some bold leap that might end in utter disaster.
Also if you look at the timeline I’m having trouble seeing any individual FDA decisions that actually caused this. Maybe something like ... the FDA makes it too frictiony to get a drug approved, and that encouraged the doctors to be slow in approving the drug and slow in encouraging the fish lipid company to move to the US / any new companies from getting a competing fish lipid approved? Because that feels VERY different from “the FDA intentionally dragged its feet and didn’t approve the drug for 15 years” whereas the treatment developers account has literally everyone dragging their feet and the FDA being helpful. It’s really hard to figure out what sort of changes you’d want when the issue is “systemic slowness and lack of interest in new drugs caused indirectly by a bunch of stuff including the FDA’s slowness in all areas”
Yes, I agree. In this follow-up post the FDA actually comes across looking like one of the few good guys, which is pretty much the complete opposite of the role Scott was suggesting for them by this very example in the previous post. It's a remarkable change, from black to white hat, which is why I think it was awesome and fascinating that Scott wrote the follow-up, and why I think it's a cautionary tale in judging too quickly.
And, well,
https://mobile.twitter.com/RichardHanania/status/1423141770494234624 “ A bunch of babies kept dying and getting permanent neurological damage, and everyone knew exactly how to stop it, but if anyone did the FDA would take away their licenses and shut them down.” In a sane world the entire FDA would be at The Hague.”
And just a lot of other people who particularly highlighted that particular passage as egregious albeit without that explicit conclusion, there are a ton of these links, but one more https://mobile.twitter.com/calebwatney/status/1423372377476538370
It seemed to have a significant impact.
> After more research, I’ve concluded that the story I told is basically true, although I got a few details wrong
I’m not sure that that sentence Richard highlighted was “basically true”? I could be misreading something but it doesn’t feel right. Caleb for instance did link the correction but Richard hasn’t yet
Obviously Scott is much less bad here than anyone else and everyone else with any popularity does much worse daily
Another note is that a lot of anti fda cost benefit analysis claims are of the form “if approval based prescription were relaxed, many of the hospitals who were prevented from prescribing the drug would use it and many of the babies who died from the lipid wouldn’t have”. But if this relies on the fda making things harder to realize drugs are good, leading to the entire medical system rejecting it, that’s harder to argue will just disappear if FDA stuff relaxes. In particular, in medicine pre FDA in the medical Wild West, I still remember a lot of instances of “new treatment worked really well but it still took ten years to convince people” so one might need to target that specifically? But also ensure you don’t get Twitter hydroxyvermectin along with it. Maybe that’s part of why they were so resistant - so many bad treatment ideas come along you stop listening to good ones? Idk.
I do want to understand how that stuff happens. Like there were a bunch of physicians who really thought that their prescribing those were saving lives, and that they were seeing the lives being saved in the patients they were caring for, who they saw. Is it just a random effect thing where one in N attempt*physician combinations will produce a differential in outcome, and then those take off?
Well, that's the Internet. I'm reminded of a great Sidney Harris cartoon, in which one management suit is showing off the brand new (big iron) computer to another: "And in 1/10,000 of a second, it can compound the programmer's error by a factor of 1 million!"
Hard to change when the institutional default is to hold informed consent in such low regard. No matter how much information we give a person, their consent isn't meaningful. Also, we can't compensate them meaningfully for their participation because, again, their viewed as having the same autonomy as insects. (They couldn't possibly resist the temptation provided by an amount of money that could actually have a palpable effect on their life!)
I want to question this: "Discoveries are made by weird tinkering"
There has definitely been a time when that was true. But I'm not sure it is any more. I'm also not sure that you can describe doing something that may well result in the death of a patient as weird tinkering. Tinkering sounds consequence-free, but medical experiments generally aren't. And when your tinkering require Harvard-level education... I'm not sure it's tinkering any more.
I love the lone tinkerer model of innovation as much as the next person, but I don't believe it....
Why? I think this comes out of my professional experience as much as anything. I'm a translator, which is something that millions of people do on an amateur basis. And those amateurs are not creative. I invent more new translations before breakfast than my tinkerer friends do in a lifetime.
So, from (1) a consequences point of view and (2) an inputs point of view, I think professionalism is a better model for creativity than tinkering.
An aspect Scott didn't go into in much detail is, why should it be this complicated and difficult for credentialed doctors to use a medicine or supplement that is used, apparently without problems, by millions of people per year? Sure, maybe the FDA wants to have its own particular standards of scientific proof before officially announcing that a medicine is safe and effective. But why **forbid** a doctor to try something only very likely to be safe, rather than extremely, if they judge that it's worth trying? After all, we already allowed this in all sorts of degrees. Doctors routinely prescribe precise schedules of dosage that haven't been explicitly tried before in an FDA approved study. They routinely prescribe combinations of drugs that haven't been evaluated in combination in an official FDA study. They also frequently prescribe drugs for uses that were never specifically evaluated by the FDA. There is some gray area here around how officially sanctioned these things are, but they are certainly not actively blocked by the FDA. So why block prescribing or importing a medicine that has been evaluated in trials with thousands of participants, by another agency that the FDA has general trust in? Wouldn't that still be an extremely high standard of evidence?
I have no idea as to the internal workings, but I do wonder if there's some pressure from the pharma companies? "We have to spend billions to test to get certified but you're letting some foreign company drug onto the market here for nothing? And they get handed our market for this illness on a silver platter? We will not stand for that!"
At the bottom of the FDA story, there's the problem of how to combine the careful approach needed to test new drugs with the somewhat reckless approach needed to do the cowboy thing. In other words, how to give researchers some slack without compromising the safety provided by the existing process.
Now, I've been a Google SRE for several years. These are the people who make google so reliable that you do ping www.google.com to check whether your network connection is working. The problem there is similar: You want a rigorous process to make sure that stupid mistakes won't happen. At the same time, you want those people to act like cowboys when the shit hits the fan.
One part of solution is the so called "blameless culture", i.e. if you screw up, you are not blamed. You won't be blamed by your colleagues or higher-ups. Your name won't leak out of Google. It will be Google taking blame for the outage, not you as a person. That gives you the slack needed to do the right thing when needed. But the blamelessness culture is complemented by, let's call it a "postmortem culture", that is, the process of incorporating what you've learned in the cowboy mode back into the formal process. It involves writing down what happened that needed the cowboy response and all the interested parties discussing how to prevent such scenarios in the future. The third component is having people on the ground deciding on the changes to the formal process, no political supervision involved.
Couple of comments:
1. The blamelessness principle also exists in politics. Some conutries, e.g. Germany, Switzerland, EU, use so called collegiality principle, where the ruling body makes collective decisions and the blame for the outcomes as a body, not as individuals. This also has some nice side effects, like putting the members of the body on the same boat and making them more prone to cooperation than to fighting each other.
2. An example of a postmortem (from the healthcare sphere): http://epmonthly.com/article/not-heroes-wear-capes-one-las-vegas-ed-saved-hundreds-lives-worst-mass-shooting-u-s-history/
3. No political supervision for process changes may sound hard to achieve in the real world, but once you give people on the ground the right to go cowboy anyway, the process becomes more of a "best practices" thing and there's little political gain in trying to influence it.
Book review request: Upton Sinclair's "The Jungle" I'd be interested to see whether there's a reason for the FDA's madness to be found in its origins, (and also to see Scott grapple with what I can only imagine is the Gilded Age equivalent of a quirky rationalist blogger).
Sinclair would be a tanky, I think, not a rationalist. (Less inflammatory, he was a leftist and was trying for more regulation and bigger, more emotionally driven government regulation.)
Does anyone know why FDA approval costs so much? What components of the process add up to a billion dollars? We need ideas which reduce the whole cost of the pipeline without compromising standards, starting with understanding where the current costs come from.
Not my field, but my vague understanding is (1) clinical trials, in particular the huge Phase III trials needed to prove efficacy, are very expensive, and (2) the cost of FDA approval is generally calculated by dividing the total research costs for all the drug candidates divided by the number that get approved.
Since almost all drug candidates fail at some stage of the process, you have to add up the cost to synthesize 10,000 early candidates, the cost to optimize 1,000 promising molecules, the cost to do animal studies on 100, the cost to do Phase I trials on 10, the cost to do Phase II and III trials on 5, and attribute all that monster cost to the 1 drug that makes it all the way through the gauntlet and gets approved. (Those numbers are totally made up, the reality is probably if anything worse.)
One way to address (1) would be to have the government pay for Phase III trials, i.e. move the subsidization from all the future patients who use the successful drugs to all the taxpayers, on the grounds that we all benefit from this kind of research. It would certainly "reduce" the outrageous cost of on-patent successful drugs, but not really, it would just shift the tab from patients who use the drug to everyone's tax bill (although it would obviously be much less per capita). But there are lots of complications there, naturally. It would definitely shift the motivations of drug companies, if they weren't risking their *own* money on Phase III trials, but whether for good or ill is hard to guess.
I'm not sure there's any good way to address (2) other than to have drug candidates fail faster and earlier, and the pharma industry already tries their best to do that, with all kinds of tricks. And you don't want to do that *too* much because then you discourage genuine innovation, and everyone spends his time doing the pharma research equivalent of making "Iron Man XVIII", an endless series of knockoffs and reruns, instead of trying something risky and new.
What if instead of the 1 successful drug pays the monster cost of 10000 candidates, 9999 failed candidates pay the 10000 in total? What you described seems like promoting spamming as many candidates as possible since you don't pay if it's not approved?
Did I misunderstood something? English is not my primary language.
The pharma companies do pay those costs. But any drug that gets through has to pay back all those costs for the company to break even. This isn’t approval cost as in “company pay the FDA all this money”, but costs as in “this is how much money a company spends on all of drug development on average per approved drug”
The successful drug is the only one that can be sold, and therefore the only drug that actually produces revenue. So its sales need to pay for *everything* the company does, including all the failed drugs.
Imagine cars had to be designed by trial-and-error, and 99 times out of 100, they would explode the first time they were driven -- *and* there was absolutely no way to know whether they would explode except by building them and trying to drive them.
So Ford pays a big team of engineers to design new cars, and unfortunately 99 times out of 100 the new cars they design explode and can't be sold, but they spend a lot of time designing and building the cars. 1 time out of 100 the car *doesn't* explode on testing, and then they can sell it. But the revenue they get from selling that 1 model of car has to pay for the costs to develop the other 99 that exploded as well. So it's a very expensive car.
I don't know if this is discussed elsewhere, I haven't read all comments but enabling cross approval with EU seems like it would at least alleviate many of the problems. In aviation for example, up until the 737Max catastrophe, EASA (European equivalent of Federal Aviation Administration) recognized approvals by FAA (and I think vice versa). By the way, there are interesting contrasts between how FAA makes life-and-death-decisions-that-has-insurance-consequences and how FDA does so. FAA, probably due to budget cuts or something, and probably also because airplanes are getting more complicated, doesn't have enough inspecting engineers to license an airplane, so it asks the company in question to kindly license the airplane they developed. This sounds scary, and it is.
Maybe it's because they don't worry as much about losing public trust? I don't know but I believe this is something to think about.
My main point was though the cross-approval between US and EU. I probably wouldn't do this with an economy which US is in a kind of economical cold war with such as China though.
The issue is if there is reciprocal approval you only need to regulatory capture one regulator.
The FDA culture still remembers thalidomide, which was approved in Europe and the UK around 1960, and caused something like 2000 infant deaths and 10,000 birth defects before it was pulled from the market. The famous Frances Kelsey as an FDA reviewer declined to approve the application for it to be marketed in the US, based in part on the fact that it was prescribed as a treatment for pregnant women *but* no studies had been done on its effect on the fetus. As a consequence of her intransigence in the face of "but 40 other countries have approved it!" the United States was almost entirely spared the consequences of thalidomide use.
EU and UK approval bodies probably have learnt from their mistakes in the last 6 decades, and even if not, the amount of lives and life quality lost now seems more than what would be lost if they miss a Thalidomide once every century or so based on how Scott writes about FDA.
They probably have, but the only way to know for sure would be to fling the gates open and see if another thalidomide happens, and I would guess they don't see any compelling reason to do that.
If European drug discovery and approval were twice as fast as the US or something, and the US was lagging seriously behind in the latest innovations, then there *would* be some kind of solid reason. But so far as I know, the general pace of approval is not that dissimilar -- if anything, it's a bit faster and earlier in the US. A quick gpogle search suggests the FDA approved about 60 new drugs in 2018 and the EMA about 40. It also appears that of the drugs approved by both the FDA and EMA, about 2/3 are approved *first* by the FDA. So it doesn't really look like in general the FDA is dragging its feet relative to the EMA, even if for this drug or that there is some discrepancy that is obnoxious to patients.
And, with all due respect to Scott, his is an amateur analysis, so while it's interesting it's only a starting point, a better analysis might not reveal the same thing. Furthermore, the question isn't just one of numerical analysis: whether it is "better" to save more lives through easier approval at the cost of the rare disaster is an ethical judgment, and reasonable men may disagree on the answer.
For what it's worth, to the extent we can discern the dominant consensus of ethical values among Americans, it does not seem especially congruent with the utilitarian calculus that just reduces it all to QALYs saved and lost.
" FAA, probably due to budget cuts or something.. asks the company in question to kindly license the airplane they developed. This sounds scary, and it is." -
An alternative to having a company review its own product is the "reviewer of reviewers" model that is used in the EU. In the EU the EMA contracts out some of the reviewing to private firms. This introduces competition and incentives to improve things into the system. I saw a paper analyzing this that I think was written by someone who is involved with Effective Altruism, but I can't remember who the author was and can't find it now! I think it was a thesis of some sort. If I stumble upon it again I'll post it here.
How is having a placebo group unethical when the placebo treatment is the current standard of care?
In this case, because the placebo group would have several dead babies that would live on to long and happy lives if they weren’t placebos. (Assuming whatever treatment it is works, which it does here
That was my initial reaction too. But if the trial is run with a placebo group it is more highly regarded, the treatment may be approved and there may be many less dead babies in the future.
This is were Utilitarianism becomes ugly. Solving for the least number of dead babies may be optimal, but at the same time it is horrible
Illustrated well in the Ursula K LeGuin short story, “Those Who Walk Away from Omselas.”
https://en.m.wikipedia.org/wiki/The_Ones_Who_Walk_Away_from_Omelas
If babies die because the standard of care is bad, that's just how things are. If you call those same babies a 'control group' you suddenly have a duty of care.
If hundreds of Germans die because there weren't enough covid vaccines, what can you do? If one dies of a blood clot after getting AZ, then the medical system has failed them.
https://blog.jaibot.com/the-copenhagen-interpretation-of-ethics/
I much appreciate this "greater truth" follow up.
I propose the Alcohol Standard: Anything less dangerous than alcohol should be legal.
If you want to make a drug legal, all you should have to do is show that it is less dangerous than alcohol. If you want it to get FDA Approved or if you want to make a drug which is more dangerous than alcohol legal, then a more complicated process should be required.
I was a pediatric intern in 2013 and can offer a small amount of insight into how TPN cholestasis and Omegaven was viewed in NICUs from 2012 when I first encountered discussions of it through to now.
As background, the TPN (total parenteral nutrition – the nutrition delivered into veins) cholestasis (lack of bile flow) has been a described complication of TPN or PPN (partial parenteral nutrition) for a very long time. It afflicts neonates who aren't able to take a sufficient amount (or potentially any) nutrients via their GI tract. It also affects older children (and some teens and adults) whose GI tracts are unusable or those who have so called "short gut syndrome" (often children who had significant portions of their small intestines removed due to an infection called necrotizing enterocolitis that is not uncommon among significantly premature infants).
In the early 2010s it was widely known that some small percentage of patients treated with TPN would go on to develop cholestasis, which was an easy thing to check by monitoring the patients' bilirubin levels. About the only identified risk factor for who would develop it was how long the patient had been solely receiving TPN.
If/when they developed TPN cholestasis there were a few options depending on the patient circumstance. In some cases you could decrease the amount of calories/day that the patient was receiving from the lipid emulsion and their cholestasis would improve. In other cases you could stop giving them the lipid emulsion component of TPN (TPN is a very complex product that has a fascinating history, it includes carbohydrates, proteins, trace minerals, vitamins – everything you need to grow and survive) for a period of days-to-weeks and then reintroduce it and often times their cholestasis would not immediately recur. In other cases it would, which usually meant that a pile of paperwork got filled out and even back in 2013-2014 you could get them compassionate use of Omegaven, which the FDA acknowledged seemed not to cause these problems. While the Omegaven saga was playing out, in fact another product was undergoing active study and trials in the US what was an option for patients to be enrolled in initially, or to also get via compassionate use – that product was SMOF lipid, which is used much more widely today than Omegaven is, and getting patients who'd failed the traditional soy lipid formulation onto it was the same paperwork but reliably effective.
The fact of the matter is that even today, in 2021 with fully approved intralipid (the old "problematic" lipid formulation, Omegaven, and SMOF-lipid) many hospitals in many circumstances still use the original intralipid formulation because it is not uniformly problematic, the problems are easy to detect in the initial stages (and they're reversible!), and the alternatives are significantly more expensive.
I'm an enormous critic of the FDA and many of the ways that it undermines appropriate research on a variety of things by giving out too many compassionate use waivers and wastes valuable time on time sensitive investigations, but the story of intravenous lipid preparations is not one of FDA failure. There were many failings of the wider medical establishment in recognizing and propelling Omegaven forward with more alacrity, but that is a totally different problem of insularity and arrogance.
This slide deck from CHLA is a good primer (though rather technically dense) on intravenous lipid preparations and has a few notes about the history https://www.chla.org/sites/default/files/atoms/files/SMOFlipids.pdf
"it undermines appropriate research on a variety of things by giving out too many compassionate use waivers"
Yes, but as a public body it is very susceptible to pressure when interested parties are feeding the media "Babies are DYING" stories. Desperate patients and their families, cowboy medical professionals and drug companies all collaborating to lobby for compassionate use waivers are hard to resist, and even if you are fully prepared to be a stony-hearted bureaucrat swathed in red tape, your political masters are not when a canny grass-roots campaign to "contact the local congressperson and ask them to sign on to this!" is in place, and Senator Braces who is the representative of the Desperate Families in question starts making noises at the administration to tell them to tell you to let this drug through.
Reading this made me realize I really don't know anything about how medical studies and regulations around them work. Why was it never a relevant question to go "Hey, do the babies over in Europe who receive Omegaven get PNALD at a lower rate?" Shouldn't that be significant enough to get you out of the "rut" of "drug is too good to make a control group justifiable, but doesn't have enough evidence behind it for people to care."
I have some understanding that the FDA prefers (or requires?) American studies, or something, so maybe this is the problem?
I'm pretty sympathetic to the need for "cowboys" in medicine, because those are the kinds of doctors who actually got transition medical care for trans people to the semi-tolerable point where it is today in some countries. It took some surprising guts and a certain amount of generational change for doctors to actually start listening to what their trans patients were saying. And improvements in HRT protocols are all done pretty "cowboy" because of the difficulties of getting any decent study funded and all the drugs involved being used off-label. The official guidelines from groups like WPATH or my local org Rainbow Health Ontario, while less bad than what was done two or three decades ago, are still pretty suboptimal in a lot of ways based on some of the preliminary stuff that comes out of clinics which are willing to experiment a bit. If you get progesterone as a trans woman (which seems to help complete breast development, improve libido, etc.), you have to sign a waiver which says "officially this does nothing, but we agree to prescribe it to you anyway", and many doctors just won't prescribe it because officially it does nothing. There's also a bunch of misinformation about the risks because the official guidance doesn't distinguish between different types of progesterone medications, some of which have significantly worse side effects than others. Repeat for a variety of other medications and combinations thereof. You need someone willing to do their own research and push the boundaries in order to get the best care.
At the same time, it's a bit of a double-edged sword. One such cowboy in the UK was trying experimental psychotherapy of dubious validity (specifically, therapy intended to make a gender-dysphoric person not transition, which has little evidence to support its effectiveness and a good amount of evidence to show that it actually causes harm, as I hope you know) on non-consenting children who called a mental health helpline about their gender troubles.
This is also why I have some fearful uncertainty about some of your suggested changes to FDA approval and drug coverage in a previous post. Because transition medications and other procedures will basically never get a good randomized controlled trial done for it for a variety of reasons, it would be hard for it to get the kind of full approval needed to force insurance coverage in your suggested system, even with cost barriers to getting certification removed, which leads to a big mess of coverage or lack thereof (coverage is already a big mess, I foresee this making it worse) and it being used by red states to score culture war points, as trans care for minors currently is.
Am I the only person who finds the statement 'Every single one of the parents of a child in the control group noticed such inferior results that they begged and pleaded with us to let them have the treatment and we being softies gave it to them so our results are weak' more than a little bit insane? Like, surely whatever early data the parents were noticing was so strong that you could have recorded it and used that as the basis for robust results? Then again, there is the catch-22 of not letting anyone do a study to prove that it's effective because it's too effective for it to be ethical to have a control group, so there's plenty of insanity to go around here.
I'm amazed that nobody has mentioned the controlled trials on the use of oxygen in premature babies. It's my understanding that the doctor who did the study knew that he was going to blind some of his subjects and possibly kill a lot of them, but that has prevented a lot of babies from being blinded since then. I don't know what it's ethical to do, but in cases where the accepted treatment is worse than doing nothing and it's almost universally used, one can make a strong case for acting.
Oh and apropos of nothing, THANK YOU for doing psychiatry. I keep thinking about how lovely that is. <3
(I mean, the people who've kept us from nuclear doom are also lovely. Not to take away from them. Keep it up. Please. )
Judges thought :(
I appreciate the deep-dive post. I actually came away from this thinking that it somewhat weakens the overall argument that the FDA is in the “wrong” in this case.
> I realize the hurdle is there for a reason.
I’d be really interested to see you flesh this out further.
I think we are covering a subject where it’s just not valid to look only at the costs of the current regime. We need to also make an effort to compare the benefits too. A zero-regulation environment (or even just a move directionally towards less regulation / faster approval) would clearly also involve harms, or put differently, the status quo is averting real harms. The question is whether for a new position on the spectrum, the new benefits would outweigh the new harms.
Thalidomide is the obvious one that the FDA remembers deeply, but I think it would be more interesting to (say) investigate the covid vaccine candidates that didn’t get through the approval pipeline, or some other drugs that were dropped at phase 3/4 due to side effects that were only detectable with very large studies. Essentially, try a steelman of the FDA’s current level of conservativity based on the public record.
Another related critique I’d make of this piece is that the drug approval process is complex enough that I’d be very suspicious of claims that you can make a localized change to fix one small scenario like “allow Omegaven to be used when it clearly works”, without having substantial knock-on effects on potential widely-used drugs that would swamp the positive utility of fixing this case. For example, would we also get statins which are more efficacious but we later discover have bad side effects? My impression from conversations with people in the field (so a low-confidence assertion) is that there are lots of drugs where the cost/benefit ratios are not that far below 1, and so undetected side effects would take it from a net benefit to a net harm. Would be interested in others’ thoughts/input on this angle. To be clear it does seem to me that there are improvements to be had, but I think we’re optimizing a U-shaped function and need to be careful not to make things worse.
There are two distinct but related aspects of human nature that make this problem fraught:
(1) Human beings are by nature optimistic[1], meaning they will generally overestimate the probability of new approaches being successful. If you leave it up to "common sense" and intuition -- even that of experts -- you will generally considerably raise the level of garbage relative to gold, if history is any guide often to the point where it becomes almost impossible to discern the signal from the noise.
(2) Human beings usually prefer to rationalize their failures than admit them. It's very difficult to say "well, I was just wrong" instead of "well, I was right -- but not at the right time/except for this weird exception/but my idea was executed properly/in principle." This is most especially true when the stakes were very high, e.g. a human life. It's extrremely difficult to say "I was wrong and someone (maybe me) died as a result." People will struggle to find *any* rationalization for why they were actually right, but some other random factor prevented their rightness from resulting in success.
Both these factors mean that medicine as a field of human knowledge has advanced shockingly slowly throughout human history despite its overwhelming importance to us. It has been held back for years, decades, centuries, by inherent optimism about one's own hypotheses, the inability to accept failure, and our inherent preference for rationalization instead.
Presumably the fact that stakes are so high is why these psychological and sociological factors have held back medicine much more than they held back, say, the development of steam engines or electronics. If you build an engine based on Theory X and it blows up, it's just a lot easier to accept your failure and learn something than if you treat 10,000 patients with Theory X and they die or suffer horribly. One would guess that is why historical events we now find horrifying -- trepanation or leuctomy for mental disease, bleeding and cupping, the failure to realize plague was carried by fleas on rats, the failure to appreciate that clean hands saved lives during operations -- went on for so long.
The post-Enlightenment solution has been to force ourselves to act like skeptical empiricists, even though it is very much not in our nature, and nobody does it naturally -- it's like being on a constant mental diet of raw broccoli and bran muffins, never allowing yourself the pleasure of the ice cream sundae of just believing what your instincts suggest is true. We insist everything, even the stuff that seems obvious, must be tested in nitpicky and even OCD detail, and we take extreme safeguards against optimism and rationalization having a role in final judgment, however fraidy-cat unimaginative heartless absurd that may intuitively assume. We try to institutionalize Murphy's Law.
The practical results are certainly impressive: the world has seen advances in science and medicine since this became the scientific norm that utterly dwarf anything accomplished in the past 40,000 years, with perhaps the exception of the invention of speech (and we did not accomplish that through conscious effort anyway, one assumes).
But we are perfectly capable of rationalizing away that success, just as people can rationalize that their weight loss and improved health came from happier thoughts from a new guru, or a new arm workout regime, rather than all the damn broccoli and bran muffins and absence of ice cream (and then abandon the bran and gain back the 15 pounds promptly, which will in turn be rationalized as the result of sleeping on the wrong pillow or not enough cinnamon).
It has been so long since we lived in the squalid world of scholasticism and alchemy that I daresay in the West we are capable of restlessly abandoning the unnatural and unpleasant strictures of Western empiricism, and taking our heritage of technological progress entirely for granted -- treating it as some kind of natural order of things, instead of a hard-won victory over our natural tendencies. That *would* be in the nature of humanity: as they say, soft times lead to soft men.
Forgot the reference:
[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4467896/
Huge fan, but I think Scott's being too easy on himself here. This was THE example that the rest of his initial FDA takedown hung on - the implication was "throw a dart at the FDA and here's what you'll find, it represents a thousand other stories just like it." It turned out to be substantially wrong. Like, AT LEAST as factually wrong than the crappy NYT story about Scott which he (correctly) didn't excuse.
Scott's charging for content now, and is doing important, high profile investigatory journalism. It's not OK to get so many relevant facts wrong. "This story is directionally and emotionally accurate, so don't worry that my supporting arguments were wrong" is exactly the kind of BS I want to see ACT/SSC as fighting against in the rest of media.
I would've liked to see a full retraction here, and most importantly, I'd like to see some of the subscriber money go to an intern who can help fact-check.
(and again, I'm a huge fan -- that I can even imagine Scott taking the high road here is the reason this disappoints me so much more than the typical BS in another publication.)
FDA has recently simplified the process for applying for "Expanded Access" to prescribe non-approved medications. FDA says they approve over 99% of applications. Maybe it is easier now compared to when you tried? https://www.fda.gov/news-events/expanded-access/expanded-access-information-physicians
Has anyone done a randomized controlled study to prove that the FDA is both safe AND effective?
If no one has done such a thing, why are we using it?
That omega-6-based nutrition causes liver issues in newborns isn't surprising, given that humans evolved consuming extremely low amounts of it.
https://www.jeffnobbs.com/posts/why-is-vegetable-oil-unhealthy
I cannot help but think of a Dr. Gura championing a fish oil treatment as having the voice of a certain shark vTuber.
> What would a still-default-no-but-the-hurdle-is-smaller medical world look like?
How about automatically approving any drug or treatment approved by at least N other first world regulatory bodies, where N is ideally equal to 1.
Okay, it's true. We'll currently fast-track applications for drugs that were approved by the British NHS, and it's already considered one of the easier ways to enter the American market ("apply in Europe first"), but that fast track could easily be an automatic reciprocal approval process among a treaty of nations (Canada, the UK, the EU, I would accept safety data from Singapore as Phase 1 trial data here though their efficacy data is all postmarketing studies). PS: I feel like the flip side of the FDA being "default no with high hurdles" on drugs is this story about the FDA continuing to permit the implantation of medical devices as evidence mounted against their safety https://www.propublica.org/article/heartware-patients-implanted-fda (and also 510(k) abuses, with the history of the vaginal mesh catastrophe, the cobalt-laden joint replacements causing bone degradation, the heart valves that caused clotting and death; those are just the ones I remember off the top of my head).
Who cares if it's reciprocal? If those other nations want to waste money re-checking already safe enough drugs, they are welcome to do so. Framing it as a treaty is just counter-productive, it can and should be a unilateral decision.
No, you're right. I tried to like that comment from the email, but I don't think it worked.
Why couldn't they compare death rates between the USA and Europe, where Omegaven was already being used? Too many confounders?