This is a side issue, but there really needs to be a single word or short phrase for "we are pretty sure this medical practice works, the existing controlled trials are not good enough, we will never be allowed to conduct the trial because no one will let us have control group". Examples:
- Diphtheria antitoxin in the early 1900s. There is a section in the book The Microbe Hunters about how the initial trial had lower than usual mortality but no control group, but now (written in 1926) we'd never refuse to give a child antitoxin so we can never do a controlled trial. So like, we're pretty sure this works but we'll never know.
- Surgical masks to block spread of disease. See Scott's posts back in March 2020.
- Apparently, Omegaven rather than soy-based competitor.
I suspect there's many others, even excluding obvious ones like "bandage wounds instead of letting people bleed to death".
Something similar happened to me, where the FDA did go faster than their normal process, but still nearly killed me in the process. Dan Elton, of the US Transhumanist Party, wrote an article about it:
What is the actual number of babies that died due to this? Definitely hundreds? (I saw a claim on Twitter it was more like “ten or twenty” than “hundreds or thousands”)
So why were they so sure that "it isn't the lipids"? Because it seems a bit less certainty on that could have helped a lot. Was that a reasonable position? Would the establishment still have the same kind of position in the same kind of situation or has something changed since then?
Curious about PNALD in Europe. This story is a little different than a new discovery of an alternate use for an older drug or something where there isn't already a body of evidence that it works. If Omegaven was already a standard of treatment there, was there a correspondingly lower incidence of PNALD? Might that have been valid data to use along the way?
It sounds like your qualm is with how American healthcare law is written, not with how the laws are administered by the FDA.
As an example, who are you more mad at: judges who upheld disparate sentencing guidelines created during the war on drugs, or the legislators who wrote the laws?
Thank you for this story of heroes and a system that can be so cruelly imperfect.
I'm glad you plan to write more on this topic in the future, because it's so important, and your analysis of it has been incredibly enlightening. Do you plan to write about the approval process in other countries as examples of ways the FDA can be improved? Or as cautionary tales?
Wait, so the only evidence for the effectiveness of fish oil in treating PNALD is a poorly powered study and a few miracle cure anecdotes? Miracle cure anecdotes are a dime a dozen. You can easily find thousands of them for everything from homeopathy to touching a statue of the Virgin Mary. That doesn't mean homeopathy obviously works or should be approved by the FDA.
"We were confident we were onto something and began to discuss our findings with others at BCH. The response was not what we expected. We were told “Everyone knows it's not the lipids because liver disease happened with them and without them.” Some individuals would even pull me aside to ask “Are the results real?”"
Usually when one scientist is convinced about the miraculous properties of their discovery while everyone else is skeptical, it's everyone else who's right, not the one scientist. I have no opinion on who's right in this case, but I wouldn't be surprised if fish oil turns out to not be very effective at all in treating PNALD.
“this drug is too good so it would be unethical to have a control group”
Around 15 years ago, my wife was taking chemotherapy as part of the regimen to treat her colon cancer. Her oncologist asked if she'd be interested in participating in a trial of a new chemo drug. (I don't recall its name or maker, sorry.) She was interested... on the assumption that she'd be in the non-placebo group.
Not long before the trial actually began, she told me they'd decided to change it so that *everyone* would get the new drug. No one with cancer wants to be in the placebo groups!
Obviously not a double-blind study anymore. I'm not sure how the maker managed that switch or what exactly their new trial design was.
> It will be really hard to change the system radically enough to make it possible to do better, but I still think we should try.
Lots of people agree with this, but the political coalitions in the US make this impossible.
If you read Charles Murray’s By the People and Krugman’s Conscience of Liberal (both MIT PhDs), they are actually 100% on the same page about who the players are and how the coalitions arose.
According to Murray, he believes the post 1950 government is illegitimate, and the last time America was still America was the 30s. And that people who think like him took over the Republican party starting with Barry Goldwater, via a few influential publications and right wing think tanks funded by a handful of wealthy families (who he thanks individually, and where he has worked most of his career).
Krugman is exactly on the same page, with the people, institutions, ideology, and timeline.
Krugman’s main counter-point is that the broad middle class that emerged after WWII was not the natural outcome of technology but massive interference in the market, and the natural market outcome is industrial-revolution like inequality. Which I am also convinced is correct for reasons too long to get into here.
An ideal state would combine the polices that maintain a less unequal distribution of income (which is probably needed to maintain political stability in democratic countries), with heavy doses of capitalism and markets. There are certainly people in the center-left that support this. But they are in a coalition with the left, because after the Republican party takeover not enough common ground exists to make policies that keep inequality in check.
So, maybe we need radical change. But with these coalitions, what radical change is possible? Where is the radical change going to come from? Until we can answer those questions adequately, we should be careful what we wish for.
Why is not having a control group considered such a deal breaker? I know it makes it hard to tell if there was a placebo effect, but if an experimental treatment is much more effective than a placebo possibly could be, shouldn't that be enough?
I think I missed something. In Europe they use fish oil based nutrition. So wouldn’t rates of PNALD be much lower in Europe? Is the advance that it has to be pure fish oil?
Amazing. This ultra obsession with randomized controlled trials is crazy -- complete brain dead. Come on I get it. If you want to pick up very small effects yes you need them, but if your expectation are big results just use some bloody common sense. You will be able to see them if they do exist. The base line is that all babies die. If you save 20/100 you have your answer in front of your eyes.
The use of double blind studies of sufficient sample size is critical. But for many kinds of large scale research, the system is breaking. Mostly because the hurdles to get through multiple studies of incrementally larger sizes, and different designs, and different standards of proof. All the while trying to weigh risks and benefits for the control and intervention group.
I'm most familiar with how to evaluate genetic effects and risk factors in large populations, This is maybe a good example of how it works in medical interventions.
There's a better, more analytically principled, and powerful approach (we use this for optimizing game settings and ad placements in my current job). It's to use (usually Bayesian) models of likelihood to probabilistically assign treatment. Like, if there is a high risk, maybe-good new formula, you'd assign it at very low probability. If it has great results (the likelihood it's better goes up) and you start prescribing it more. As the sample size goes up, but the results begin looking worse, you assign it less.
Importantly, you don't need to give it to thousands of people off the bat and risk so many lives. But you can ramp it up quickly as it becomes more obvious that not doing so risks lives.
I very much want to hear about what the default-yes and default-no-but-with-smaller-hurdles models look like to you! *refreshes inbox excitedly for those posts *
I feel like it may be worth noting as a small aside here that soybean oil consumption has increased significantly in the US in the last several decades and is put in, well, just about everything I look at here. Meanwhile, fish oil is not only much less common in our standard diet, but so uncommon that it is taken as a dietary supplement by millions so that they get a small amount instead of almost zero.
I think it's likely the case that at least one, if not more, of the following is true:
1) excessive soybean oil (and similar oils, perhaps high in linoleic acid) consumption is very common and very bad for you
2) same as the above, except that it is primary bad for you when used at high temperatures such as for frying
3) the ratio of omega 3 to omega 6 fatty acids in our diet is terrible (with most of us having far too much omega 6 in comparison), partly as a consequence of the large prevalence of soybean oil and large lack of omega 3 oils such as many fish oils, and this is probably very bad for us. This may also be why fish oil supplementation seems to help some groups of people while doing nothing for other groups of people
I'm still only moderately confident in some of these, but they do seem to be relatively convincing propositions the more evidence I find in their favor, and soybean oil is probably not the best oil we could be giving to an infant regardless here, so I kind of frowned when I read that it was the standard, when it is also the first ingredient in a lot of pretty terrible things. Conveniently, it is also extremely cheap, so perhaps I should not be surprised.
"One small weak randomized trial wasn’t enough evidence for anyone else to care. But it was enough evidence that BCH refused to do larger studies, because that would require putting some babies in the control group, and obviously those babies were going to die, and that would be unethical."
It's a heart-warming story and I am very glad that now there is a proper feeding regime for babies like this.
But the bit about deliberately putting pressure on the pharma company by feeding negative PR to the media? That's how you get Aduhelm and drugs like it approved.
And the praise of cowboys - the reason "cowboy" became a term of opprobrium is because people have experiences of such types. Authoritarian surgeons who think they are 'saving lives' have caused suffering - there's one notorious scandal from the late 90s in Irish medical history:
Wouldn’t a better way forward be to grant medicines approved in reputable places like the EU, UK, Japan or Australia an automatic provisional authorization?
Even if you count from the first study in humans to the ultimate clinical trial(s), total time is 9.1 years.
But that isn't because of the FDA per se, it's because biology is really hard, and we don't want to kill people. It isn't trivial to introduce a drug to humans for the first time, as the TGN1412 study tragically showed: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2964774/
> I realize the hurdle is there for a reason. What would a default-yes medical world look like? What would a still-default-no-but-the-hurdle-is-smaller medical world look like? I have thoughts about this which I hope to address in future posts.
Given the history behind the term "snake oil salesman" and Coca(aine)-Cola, my presumption is that there used to be such a world. What were the events that led to the reversal? What lies behind Chesterton's Fence ought to be historical record, at least in part, right?
I clicked Scott’s link to the SSC post on the research study he had been doing that was swamped by the IRB. Very interesting. Here is an article that concluded what you were suspecting.
Cerimele, Chwastiak, Dodson & Katon, 2013, “The prevalence of bipolar disorder in primary care patients with depression or other psychiatric complaints: a systematic review.” Psychosomatics, 54(6).
Screener - 20.9-30.8% prevalence. Full diagnostic interview- 3.4-9%. Whee.
Have not fully read this but I guess primary care has the advantage of repeat patient contact and permanent staff.
This is a really excellent post, and I commend your willingness to dig deeper into the story and add a thorough and thoughtful correction to your original story. It was instructive to read this, not only in terms of learning what actually happened, but in seeing how the story was oversimplified and in some cases distorted over the years and among a wider audience -- which is alas all too common.
Indeed, I think one of the obvious conclusions to be drawn here is that when it seems like some large group of pretty smart people has done something just so incredibly brain-dead that it only takes a moment's thought to realize how they went wrong, and how it could have been done better, that some uncomfortably high percentage of the time that conclusion is flawed -- that there is much more to the story than was at first apparent.
Of course, sometimes people, even very smart and experienced people, do make stupid or corrupt or evil decisions, so we can't rule it out all the time. Still...if I had to make a choice, I'd say the modern era could usefully move its collective needle towards patience and away from quick judgment.
Hard to change when the institutional default is to hold informed consent in such low regard. No matter how much information we give a person, their consent isn't meaningful. Also, we can't compensate them meaningfully for their participation because, again, their viewed as having the same autonomy as insects. (They couldn't possibly resist the temptation provided by an amount of money that could actually have a palpable effect on their life!)
I want to question this: "Discoveries are made by weird tinkering"
There has definitely been a time when that was true. But I'm not sure it is any more. I'm also not sure that you can describe doing something that may well result in the death of a patient as weird tinkering. Tinkering sounds consequence-free, but medical experiments generally aren't. And when your tinkering require Harvard-level education... I'm not sure it's tinkering any more.
I love the lone tinkerer model of innovation as much as the next person, but I don't believe it....
Why? I think this comes out of my professional experience as much as anything. I'm a translator, which is something that millions of people do on an amateur basis. And those amateurs are not creative. I invent more new translations before breakfast than my tinkerer friends do in a lifetime.
So, from (1) a consequences point of view and (2) an inputs point of view, I think professionalism is a better model for creativity than tinkering.
An aspect Scott didn't go into in much detail is, why should it be this complicated and difficult for credentialed doctors to use a medicine or supplement that is used, apparently without problems, by millions of people per year? Sure, maybe the FDA wants to have its own particular standards of scientific proof before officially announcing that a medicine is safe and effective. But why **forbid** a doctor to try something only very likely to be safe, rather than extremely, if they judge that it's worth trying? After all, we already allowed this in all sorts of degrees. Doctors routinely prescribe precise schedules of dosage that haven't been explicitly tried before in an FDA approved study. They routinely prescribe combinations of drugs that haven't been evaluated in combination in an official FDA study. They also frequently prescribe drugs for uses that were never specifically evaluated by the FDA. There is some gray area here around how officially sanctioned these things are, but they are certainly not actively blocked by the FDA. So why block prescribing or importing a medicine that has been evaluated in trials with thousands of participants, by another agency that the FDA has general trust in? Wouldn't that still be an extremely high standard of evidence?
At the bottom of the FDA story, there's the problem of how to combine the careful approach needed to test new drugs with the somewhat reckless approach needed to do the cowboy thing. In other words, how to give researchers some slack without compromising the safety provided by the existing process.
Now, I've been a Google SRE for several years. These are the people who make google so reliable that you do ping www.google.com to check whether your network connection is working. The problem there is similar: You want a rigorous process to make sure that stupid mistakes won't happen. At the same time, you want those people to act like cowboys when the shit hits the fan.
One part of solution is the so called "blameless culture", i.e. if you screw up, you are not blamed. You won't be blamed by your colleagues or higher-ups. Your name won't leak out of Google. It will be Google taking blame for the outage, not you as a person. That gives you the slack needed to do the right thing when needed. But the blamelessness culture is complemented by, let's call it a "postmortem culture", that is, the process of incorporating what you've learned in the cowboy mode back into the formal process. It involves writing down what happened that needed the cowboy response and all the interested parties discussing how to prevent such scenarios in the future. The third component is having people on the ground deciding on the changes to the formal process, no political supervision involved.
Couple of comments:
1. The blamelessness principle also exists in politics. Some conutries, e.g. Germany, Switzerland, EU, use so called collegiality principle, where the ruling body makes collective decisions and the blame for the outcomes as a body, not as individuals. This also has some nice side effects, like putting the members of the body on the same boat and making them more prone to cooperation than to fighting each other.
3. No political supervision for process changes may sound hard to achieve in the real world, but once you give people on the ground the right to go cowboy anyway, the process becomes more of a "best practices" thing and there's little political gain in trying to influence it.
Book review request: Upton Sinclair's "The Jungle" I'd be interested to see whether there's a reason for the FDA's madness to be found in its origins, (and also to see Scott grapple with what I can only imagine is the Gilded Age equivalent of a quirky rationalist blogger).
Does anyone know why FDA approval costs so much? What components of the process add up to a billion dollars? We need ideas which reduce the whole cost of the pipeline without compromising standards, starting with understanding where the current costs come from.
I don't know if this is discussed elsewhere, I haven't read all comments but enabling cross approval with EU seems like it would at least alleviate many of the problems. In aviation for example, up until the 737Max catastrophe, EASA (European equivalent of Federal Aviation Administration) recognized approvals by FAA (and I think vice versa). By the way, there are interesting contrasts between how FAA makes life-and-death-decisions-that-has-insurance-consequences and how FDA does so. FAA, probably due to budget cuts or something, and probably also because airplanes are getting more complicated, doesn't have enough inspecting engineers to license an airplane, so it asks the company in question to kindly license the airplane they developed. This sounds scary, and it is.
Maybe it's because they don't worry as much about losing public trust? I don't know but I believe this is something to think about.
My main point was though the cross-approval between US and EU. I probably wouldn't do this with an economy which US is in a kind of economical cold war with such as China though.
I propose the Alcohol Standard: Anything less dangerous than alcohol should be legal.
If you want to make a drug legal, all you should have to do is show that it is less dangerous than alcohol. If you want it to get FDA Approved or if you want to make a drug which is more dangerous than alcohol legal, then a more complicated process should be required.
I was a pediatric intern in 2013 and can offer a small amount of insight into how TPN cholestasis and Omegaven was viewed in NICUs from 2012 when I first encountered discussions of it through to now.
As background, the TPN (total parenteral nutrition – the nutrition delivered into veins) cholestasis (lack of bile flow) has been a described complication of TPN or PPN (partial parenteral nutrition) for a very long time. It afflicts neonates who aren't able to take a sufficient amount (or potentially any) nutrients via their GI tract. It also affects older children (and some teens and adults) whose GI tracts are unusable or those who have so called "short gut syndrome" (often children who had significant portions of their small intestines removed due to an infection called necrotizing enterocolitis that is not uncommon among significantly premature infants).
In the early 2010s it was widely known that some small percentage of patients treated with TPN would go on to develop cholestasis, which was an easy thing to check by monitoring the patients' bilirubin levels. About the only identified risk factor for who would develop it was how long the patient had been solely receiving TPN.
If/when they developed TPN cholestasis there were a few options depending on the patient circumstance. In some cases you could decrease the amount of calories/day that the patient was receiving from the lipid emulsion and their cholestasis would improve. In other cases you could stop giving them the lipid emulsion component of TPN (TPN is a very complex product that has a fascinating history, it includes carbohydrates, proteins, trace minerals, vitamins – everything you need to grow and survive) for a period of days-to-weeks and then reintroduce it and often times their cholestasis would not immediately recur. In other cases it would, which usually meant that a pile of paperwork got filled out and even back in 2013-2014 you could get them compassionate use of Omegaven, which the FDA acknowledged seemed not to cause these problems. While the Omegaven saga was playing out, in fact another product was undergoing active study and trials in the US what was an option for patients to be enrolled in initially, or to also get via compassionate use – that product was SMOF lipid, which is used much more widely today than Omegaven is, and getting patients who'd failed the traditional soy lipid formulation onto it was the same paperwork but reliably effective.
The fact of the matter is that even today, in 2021 with fully approved intralipid (the old "problematic" lipid formulation, Omegaven, and SMOF-lipid) many hospitals in many circumstances still use the original intralipid formulation because it is not uniformly problematic, the problems are easy to detect in the initial stages (and they're reversible!), and the alternatives are significantly more expensive.
I'm an enormous critic of the FDA and many of the ways that it undermines appropriate research on a variety of things by giving out too many compassionate use waivers and wastes valuable time on time sensitive investigations, but the story of intravenous lipid preparations is not one of FDA failure. There were many failings of the wider medical establishment in recognizing and propelling Omegaven forward with more alacrity, but that is a totally different problem of insularity and arrogance.
Reading this made me realize I really don't know anything about how medical studies and regulations around them work. Why was it never a relevant question to go "Hey, do the babies over in Europe who receive Omegaven get PNALD at a lower rate?" Shouldn't that be significant enough to get you out of the "rut" of "drug is too good to make a control group justifiable, but doesn't have enough evidence behind it for people to care."
I have some understanding that the FDA prefers (or requires?) American studies, or something, so maybe this is the problem?
I'm pretty sympathetic to the need for "cowboys" in medicine, because those are the kinds of doctors who actually got transition medical care for trans people to the semi-tolerable point where it is today in some countries. It took some surprising guts and a certain amount of generational change for doctors to actually start listening to what their trans patients were saying. And improvements in HRT protocols are all done pretty "cowboy" because of the difficulties of getting any decent study funded and all the drugs involved being used off-label. The official guidelines from groups like WPATH or my local org Rainbow Health Ontario, while less bad than what was done two or three decades ago, are still pretty suboptimal in a lot of ways based on some of the preliminary stuff that comes out of clinics which are willing to experiment a bit. If you get progesterone as a trans woman (which seems to help complete breast development, improve libido, etc.), you have to sign a waiver which says "officially this does nothing, but we agree to prescribe it to you anyway", and many doctors just won't prescribe it because officially it does nothing. There's also a bunch of misinformation about the risks because the official guidance doesn't distinguish between different types of progesterone medications, some of which have significantly worse side effects than others. Repeat for a variety of other medications and combinations thereof. You need someone willing to do their own research and push the boundaries in order to get the best care.
At the same time, it's a bit of a double-edged sword. One such cowboy in the UK was trying experimental psychotherapy of dubious validity (specifically, therapy intended to make a gender-dysphoric person not transition, which has little evidence to support its effectiveness and a good amount of evidence to show that it actually causes harm, as I hope you know) on non-consenting children who called a mental health helpline about their gender troubles.
This is also why I have some fearful uncertainty about some of your suggested changes to FDA approval and drug coverage in a previous post. Because transition medications and other procedures will basically never get a good randomized controlled trial done for it for a variety of reasons, it would be hard for it to get the kind of full approval needed to force insurance coverage in your suggested system, even with cost barriers to getting certification removed, which leads to a big mess of coverage or lack thereof (coverage is already a big mess, I foresee this making it worse) and it being used by red states to score culture war points, as trans care for minors currently is.
Am I the only person who finds the statement 'Every single one of the parents of a child in the control group noticed such inferior results that they begged and pleaded with us to let them have the treatment and we being softies gave it to them so our results are weak' more than a little bit insane? Like, surely whatever early data the parents were noticing was so strong that you could have recorded it and used that as the basis for robust results? Then again, there is the catch-22 of not letting anyone do a study to prove that it's effective because it's too effective for it to be ethical to have a control group, so there's plenty of insanity to go around here.
I'm amazed that nobody has mentioned the controlled trials on the use of oxygen in premature babies. It's my understanding that the doctor who did the study knew that he was going to blind some of his subjects and possibly kill a lot of them, but that has prevented a lot of babies from being blinded since then. I don't know what it's ethical to do, but in cases where the accepted treatment is worse than doing nothing and it's almost universally used, one can make a strong case for acting.
I appreciate the deep-dive post. I actually came away from this thinking that it somewhat weakens the overall argument that the FDA is in the “wrong” in this case.
> I realize the hurdle is there for a reason.
I’d be really interested to see you flesh this out further.
I think we are covering a subject where it’s just not valid to look only at the costs of the current regime. We need to also make an effort to compare the benefits too. A zero-regulation environment (or even just a move directionally towards less regulation / faster approval) would clearly also involve harms, or put differently, the status quo is averting real harms. The question is whether for a new position on the spectrum, the new benefits would outweigh the new harms.
Thalidomide is the obvious one that the FDA remembers deeply, but I think it would be more interesting to (say) investigate the covid vaccine candidates that didn’t get through the approval pipeline, or some other drugs that were dropped at phase 3/4 due to side effects that were only detectable with very large studies. Essentially, try a steelman of the FDA’s current level of conservativity based on the public record.
Another related critique I’d make of this piece is that the drug approval process is complex enough that I’d be very suspicious of claims that you can make a localized change to fix one small scenario like “allow Omegaven to be used when it clearly works”, without having substantial knock-on effects on potential widely-used drugs that would swamp the positive utility of fixing this case. For example, would we also get statins which are more efficacious but we later discover have bad side effects? My impression from conversations with people in the field (so a low-confidence assertion) is that there are lots of drugs where the cost/benefit ratios are not that far below 1, and so undetected side effects would take it from a net benefit to a net harm. Would be interested in others’ thoughts/input on this angle. To be clear it does seem to me that there are improvements to be had, but I think we’re optimizing a U-shaped function and need to be careful not to make things worse.
Huge fan, but I think Scott's being too easy on himself here. This was THE example that the rest of his initial FDA takedown hung on - the implication was "throw a dart at the FDA and here's what you'll find, it represents a thousand other stories just like it." It turned out to be substantially wrong. Like, AT LEAST as factually wrong than the crappy NYT story about Scott which he (correctly) didn't excuse.
Scott's charging for content now, and is doing important, high profile investigatory journalism. It's not OK to get so many relevant facts wrong. "This story is directionally and emotionally accurate, so don't worry that my supporting arguments were wrong" is exactly the kind of BS I want to see ACT/SSC as fighting against in the rest of media.
I would've liked to see a full retraction here, and most importantly, I'd like to see some of the subscriber money go to an intern who can help fact-check.
(and again, I'm a huge fan -- that I can even imagine Scott taking the high road here is the reason this disappoints me so much more than the typical BS in another publication.)
Typo: "...in this country. briefly..."
"I can’t find good sources about whether the issue is that soybean oil *prevents* the liver problem, or that fish oil solves the liver problem."
There is a word wrong in this sentence, or I fail comprehension. I think "prevents" was intended to be "causes", but I'm not sure.
"...decided to do the reasonable thing and walk my patient through the process of ordering it illegally on the Internet."
While this is very cool and right, I'm concerned about the advisability of admitting it publicly
This is a side issue, but there really needs to be a single word or short phrase for "we are pretty sure this medical practice works, the existing controlled trials are not good enough, we will never be allowed to conduct the trial because no one will let us have control group". Examples:
- Diphtheria antitoxin in the early 1900s. There is a section in the book The Microbe Hunters about how the initial trial had lower than usual mortality but no control group, but now (written in 1926) we'd never refuse to give a child antitoxin so we can never do a controlled trial. So like, we're pretty sure this works but we'll never know.
- Surgical masks to block spread of disease. See Scott's posts back in March 2020.
- Apparently, Omegaven rather than soy-based competitor.
I suspect there's many others, even excluding obvious ones like "bandage wounds instead of letting people bleed to death".
Something similar happened to me, where the FDA did go faster than their normal process, but still nearly killed me in the process. Dan Elton, of the US Transhumanist Party, wrote an article about it:
https://moreisdifferent.substack.com/p/the-fda-almost-killed-me
What is the actual number of babies that died due to this? Definitely hundreds? (I saw a claim on Twitter it was more like “ten or twenty” than “hundreds or thousands”)
So why were they so sure that "it isn't the lipids"? Because it seems a bit less certainty on that could have helped a lot. Was that a reasonable position? Would the establishment still have the same kind of position in the same kind of situation or has something changed since then?
Curious about PNALD in Europe. This story is a little different than a new discovery of an alternate use for an older drug or something where there isn't already a body of evidence that it works. If Omegaven was already a standard of treatment there, was there a correspondingly lower incidence of PNALD? Might that have been valid data to use along the way?
Cowboy is still mostly a compliment at a certain hard-charging aerospace company, from my experience.
It sounds like your qualm is with how American healthcare law is written, not with how the laws are administered by the FDA.
As an example, who are you more mad at: judges who upheld disparate sentencing guidelines created during the war on drugs, or the legislators who wrote the laws?
Thank you for this story of heroes and a system that can be so cruelly imperfect.
I'm glad you plan to write more on this topic in the future, because it's so important, and your analysis of it has been incredibly enlightening. Do you plan to write about the approval process in other countries as examples of ways the FDA can be improved? Or as cautionary tales?
Wait, so the only evidence for the effectiveness of fish oil in treating PNALD is a poorly powered study and a few miracle cure anecdotes? Miracle cure anecdotes are a dime a dozen. You can easily find thousands of them for everything from homeopathy to touching a statue of the Virgin Mary. That doesn't mean homeopathy obviously works or should be approved by the FDA.
"We were confident we were onto something and began to discuss our findings with others at BCH. The response was not what we expected. We were told “Everyone knows it's not the lipids because liver disease happened with them and without them.” Some individuals would even pull me aside to ask “Are the results real?”"
Usually when one scientist is convinced about the miraculous properties of their discovery while everyone else is skeptical, it's everyone else who's right, not the one scientist. I have no opinion on who's right in this case, but I wouldn't be surprised if fish oil turns out to not be very effective at all in treating PNALD.
“this drug is too good so it would be unethical to have a control group”
Around 15 years ago, my wife was taking chemotherapy as part of the regimen to treat her colon cancer. Her oncologist asked if she'd be interested in participating in a trial of a new chemo drug. (I don't recall its name or maker, sorry.) She was interested... on the assumption that she'd be in the non-placebo group.
Not long before the trial actually began, she told me they'd decided to change it so that *everyone* would get the new drug. No one with cancer wants to be in the placebo groups!
Obviously not a double-blind study anymore. I'm not sure how the maker managed that switch or what exactly their new trial design was.
> It will be really hard to change the system radically enough to make it possible to do better, but I still think we should try.
Lots of people agree with this, but the political coalitions in the US make this impossible.
If you read Charles Murray’s By the People and Krugman’s Conscience of Liberal (both MIT PhDs), they are actually 100% on the same page about who the players are and how the coalitions arose.
According to Murray, he believes the post 1950 government is illegitimate, and the last time America was still America was the 30s. And that people who think like him took over the Republican party starting with Barry Goldwater, via a few influential publications and right wing think tanks funded by a handful of wealthy families (who he thanks individually, and where he has worked most of his career).
Krugman is exactly on the same page, with the people, institutions, ideology, and timeline.
Krugman’s main counter-point is that the broad middle class that emerged after WWII was not the natural outcome of technology but massive interference in the market, and the natural market outcome is industrial-revolution like inequality. Which I am also convinced is correct for reasons too long to get into here.
An ideal state would combine the polices that maintain a less unequal distribution of income (which is probably needed to maintain political stability in democratic countries), with heavy doses of capitalism and markets. There are certainly people in the center-left that support this. But they are in a coalition with the left, because after the Republican party takeover not enough common ground exists to make policies that keep inequality in check.
So, maybe we need radical change. But with these coalitions, what radical change is possible? Where is the radical change going to come from? Until we can answer those questions adequately, we should be careful what we wish for.
It doesn't take 10 years to hear back from the FDA after filing Scott. PDUFA dates are within 10 months of filing an NDA
Why is not having a control group considered such a deal breaker? I know it makes it hard to tell if there was a placebo effect, but if an experimental treatment is much more effective than a placebo possibly could be, shouldn't that be enough?
I think I missed something. In Europe they use fish oil based nutrition. So wouldn’t rates of PNALD be much lower in Europe? Is the advance that it has to be pure fish oil?
Amazing. This ultra obsession with randomized controlled trials is crazy -- complete brain dead. Come on I get it. If you want to pick up very small effects yes you need them, but if your expectation are big results just use some bloody common sense. You will be able to see them if they do exist. The base line is that all babies die. If you save 20/100 you have your answer in front of your eyes.
The use of double blind studies of sufficient sample size is critical. But for many kinds of large scale research, the system is breaking. Mostly because the hurdles to get through multiple studies of incrementally larger sizes, and different designs, and different standards of proof. All the while trying to weigh risks and benefits for the control and intervention group.
I'm most familiar with how to evaluate genetic effects and risk factors in large populations, This is maybe a good example of how it works in medical interventions.
There's a better, more analytically principled, and powerful approach (we use this for optimizing game settings and ad placements in my current job). It's to use (usually Bayesian) models of likelihood to probabilistically assign treatment. Like, if there is a high risk, maybe-good new formula, you'd assign it at very low probability. If it has great results (the likelihood it's better goes up) and you start prescribing it more. As the sample size goes up, but the results begin looking worse, you assign it less.
Importantly, you don't need to give it to thousands of people off the bat and risk so many lives. But you can ramp it up quickly as it becomes more obvious that not doing so risks lives.
I very much want to hear about what the default-yes and default-no-but-with-smaller-hurdles models look like to you! *refreshes inbox excitedly for those posts *
FYI, Eliezer wrote about that story in 2017.
https://www.lesswrong.com/posts/x5ASTMPKPowLKpLpZ/moloch-s-toolbox-1-2
Man, the timeline on this patent tells a story. https://patents.google.com/patent/US20060127491A1/en
I feel like it may be worth noting as a small aside here that soybean oil consumption has increased significantly in the US in the last several decades and is put in, well, just about everything I look at here. Meanwhile, fish oil is not only much less common in our standard diet, but so uncommon that it is taken as a dietary supplement by millions so that they get a small amount instead of almost zero.
I think it's likely the case that at least one, if not more, of the following is true:
1) excessive soybean oil (and similar oils, perhaps high in linoleic acid) consumption is very common and very bad for you
2) same as the above, except that it is primary bad for you when used at high temperatures such as for frying
3) the ratio of omega 3 to omega 6 fatty acids in our diet is terrible (with most of us having far too much omega 6 in comparison), partly as a consequence of the large prevalence of soybean oil and large lack of omega 3 oils such as many fish oils, and this is probably very bad for us. This may also be why fish oil supplementation seems to help some groups of people while doing nothing for other groups of people
I'm still only moderately confident in some of these, but they do seem to be relatively convincing propositions the more evidence I find in their favor, and soybean oil is probably not the best oil we could be giving to an infant regardless here, so I kind of frowned when I read that it was the standard, when it is also the first ingredient in a lot of pretty terrible things. Conveniently, it is also extremely cheap, so perhaps I should not be surprised.
"One small weak randomized trial wasn’t enough evidence for anyone else to care. But it was enough evidence that BCH refused to do larger studies, because that would require putting some babies in the control group, and obviously those babies were going to die, and that would be unethical."
Wonder how many drugs are stuck in this dilemma.
It's a heart-warming story and I am very glad that now there is a proper feeding regime for babies like this.
But the bit about deliberately putting pressure on the pharma company by feeding negative PR to the media? That's how you get Aduhelm and drugs like it approved.
And the praise of cowboys - the reason "cowboy" became a term of opprobrium is because people have experiences of such types. Authoritarian surgeons who think they are 'saving lives' have caused suffering - there's one notorious scandal from the late 90s in Irish medical history:
https://www.irishexaminer.com/news/arid-30348600.html
So when it works out, great! But many times it doesn't work out.
Wouldn’t a better way forward be to grant medicines approved in reputable places like the EU, UK, Japan or Australia an automatic provisional authorization?
That Dr. Folkman sounds like a real mensch
Not sure why the post says it takes 10 years to clear the FDA. Omegaven was actually approved in under 8 months. https://www.accessdata.fda.gov/drugsatfda_docs/nda/2018/210589Orig1s000Approv.pdf
And in 2018, the "median review time was 10.1 months for
standard applications (n=79) and 7.6 months for priority applications (n=43)" https://jamanetwork.com/journals/jama/article-abstract/2758605
Even if you count from the first study in humans to the ultimate clinical trial(s), total time is 9.1 years.
But that isn't because of the FDA per se, it's because biology is really hard, and we don't want to kill people. It isn't trivial to introduce a drug to humans for the first time, as the TGN1412 study tragically showed: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2964774/
> I realize the hurdle is there for a reason. What would a default-yes medical world look like? What would a still-default-no-but-the-hurdle-is-smaller medical world look like? I have thoughts about this which I hope to address in future posts.
Given the history behind the term "snake oil salesman" and Coca(aine)-Cola, my presumption is that there used to be such a world. What were the events that led to the reversal? What lies behind Chesterton's Fence ought to be historical record, at least in part, right?
I clicked Scott’s link to the SSC post on the research study he had been doing that was swamped by the IRB. Very interesting. Here is an article that concluded what you were suspecting.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3830717/
Cerimele, Chwastiak, Dodson & Katon, 2013, “The prevalence of bipolar disorder in primary care patients with depression or other psychiatric complaints: a systematic review.” Psychosomatics, 54(6).
Screener - 20.9-30.8% prevalence. Full diagnostic interview- 3.4-9%. Whee.
Have not fully read this but I guess primary care has the advantage of repeat patient contact and permanent staff.
This is a really excellent post, and I commend your willingness to dig deeper into the story and add a thorough and thoughtful correction to your original story. It was instructive to read this, not only in terms of learning what actually happened, but in seeing how the story was oversimplified and in some cases distorted over the years and among a wider audience -- which is alas all too common.
Indeed, I think one of the obvious conclusions to be drawn here is that when it seems like some large group of pretty smart people has done something just so incredibly brain-dead that it only takes a moment's thought to realize how they went wrong, and how it could have been done better, that some uncomfortably high percentage of the time that conclusion is flawed -- that there is much more to the story than was at first apparent.
Of course, sometimes people, even very smart and experienced people, do make stupid or corrupt or evil decisions, so we can't rule it out all the time. Still...if I had to make a choice, I'd say the modern era could usefully move its collective needle towards patience and away from quick judgment.
Hard to change when the institutional default is to hold informed consent in such low regard. No matter how much information we give a person, their consent isn't meaningful. Also, we can't compensate them meaningfully for their participation because, again, their viewed as having the same autonomy as insects. (They couldn't possibly resist the temptation provided by an amount of money that could actually have a palpable effect on their life!)
I want to question this: "Discoveries are made by weird tinkering"
There has definitely been a time when that was true. But I'm not sure it is any more. I'm also not sure that you can describe doing something that may well result in the death of a patient as weird tinkering. Tinkering sounds consequence-free, but medical experiments generally aren't. And when your tinkering require Harvard-level education... I'm not sure it's tinkering any more.
I love the lone tinkerer model of innovation as much as the next person, but I don't believe it....
Why? I think this comes out of my professional experience as much as anything. I'm a translator, which is something that millions of people do on an amateur basis. And those amateurs are not creative. I invent more new translations before breakfast than my tinkerer friends do in a lifetime.
So, from (1) a consequences point of view and (2) an inputs point of view, I think professionalism is a better model for creativity than tinkering.
An aspect Scott didn't go into in much detail is, why should it be this complicated and difficult for credentialed doctors to use a medicine or supplement that is used, apparently without problems, by millions of people per year? Sure, maybe the FDA wants to have its own particular standards of scientific proof before officially announcing that a medicine is safe and effective. But why **forbid** a doctor to try something only very likely to be safe, rather than extremely, if they judge that it's worth trying? After all, we already allowed this in all sorts of degrees. Doctors routinely prescribe precise schedules of dosage that haven't been explicitly tried before in an FDA approved study. They routinely prescribe combinations of drugs that haven't been evaluated in combination in an official FDA study. They also frequently prescribe drugs for uses that were never specifically evaluated by the FDA. There is some gray area here around how officially sanctioned these things are, but they are certainly not actively blocked by the FDA. So why block prescribing or importing a medicine that has been evaluated in trials with thousands of participants, by another agency that the FDA has general trust in? Wouldn't that still be an extremely high standard of evidence?
At the bottom of the FDA story, there's the problem of how to combine the careful approach needed to test new drugs with the somewhat reckless approach needed to do the cowboy thing. In other words, how to give researchers some slack without compromising the safety provided by the existing process.
Now, I've been a Google SRE for several years. These are the people who make google so reliable that you do ping www.google.com to check whether your network connection is working. The problem there is similar: You want a rigorous process to make sure that stupid mistakes won't happen. At the same time, you want those people to act like cowboys when the shit hits the fan.
One part of solution is the so called "blameless culture", i.e. if you screw up, you are not blamed. You won't be blamed by your colleagues or higher-ups. Your name won't leak out of Google. It will be Google taking blame for the outage, not you as a person. That gives you the slack needed to do the right thing when needed. But the blamelessness culture is complemented by, let's call it a "postmortem culture", that is, the process of incorporating what you've learned in the cowboy mode back into the formal process. It involves writing down what happened that needed the cowboy response and all the interested parties discussing how to prevent such scenarios in the future. The third component is having people on the ground deciding on the changes to the formal process, no political supervision involved.
Couple of comments:
1. The blamelessness principle also exists in politics. Some conutries, e.g. Germany, Switzerland, EU, use so called collegiality principle, where the ruling body makes collective decisions and the blame for the outcomes as a body, not as individuals. This also has some nice side effects, like putting the members of the body on the same boat and making them more prone to cooperation than to fighting each other.
2. An example of a postmortem (from the healthcare sphere): http://epmonthly.com/article/not-heroes-wear-capes-one-las-vegas-ed-saved-hundreds-lives-worst-mass-shooting-u-s-history/
3. No political supervision for process changes may sound hard to achieve in the real world, but once you give people on the ground the right to go cowboy anyway, the process becomes more of a "best practices" thing and there's little political gain in trying to influence it.
Book review request: Upton Sinclair's "The Jungle" I'd be interested to see whether there's a reason for the FDA's madness to be found in its origins, (and also to see Scott grapple with what I can only imagine is the Gilded Age equivalent of a quirky rationalist blogger).
Does anyone know why FDA approval costs so much? What components of the process add up to a billion dollars? We need ideas which reduce the whole cost of the pipeline without compromising standards, starting with understanding where the current costs come from.
I don't know if this is discussed elsewhere, I haven't read all comments but enabling cross approval with EU seems like it would at least alleviate many of the problems. In aviation for example, up until the 737Max catastrophe, EASA (European equivalent of Federal Aviation Administration) recognized approvals by FAA (and I think vice versa). By the way, there are interesting contrasts between how FAA makes life-and-death-decisions-that-has-insurance-consequences and how FDA does so. FAA, probably due to budget cuts or something, and probably also because airplanes are getting more complicated, doesn't have enough inspecting engineers to license an airplane, so it asks the company in question to kindly license the airplane they developed. This sounds scary, and it is.
Maybe it's because they don't worry as much about losing public trust? I don't know but I believe this is something to think about.
My main point was though the cross-approval between US and EU. I probably wouldn't do this with an economy which US is in a kind of economical cold war with such as China though.
How is having a placebo group unethical when the placebo treatment is the current standard of care?
I much appreciate this "greater truth" follow up.
I propose the Alcohol Standard: Anything less dangerous than alcohol should be legal.
If you want to make a drug legal, all you should have to do is show that it is less dangerous than alcohol. If you want it to get FDA Approved or if you want to make a drug which is more dangerous than alcohol legal, then a more complicated process should be required.
I was a pediatric intern in 2013 and can offer a small amount of insight into how TPN cholestasis and Omegaven was viewed in NICUs from 2012 when I first encountered discussions of it through to now.
As background, the TPN (total parenteral nutrition – the nutrition delivered into veins) cholestasis (lack of bile flow) has been a described complication of TPN or PPN (partial parenteral nutrition) for a very long time. It afflicts neonates who aren't able to take a sufficient amount (or potentially any) nutrients via their GI tract. It also affects older children (and some teens and adults) whose GI tracts are unusable or those who have so called "short gut syndrome" (often children who had significant portions of their small intestines removed due to an infection called necrotizing enterocolitis that is not uncommon among significantly premature infants).
In the early 2010s it was widely known that some small percentage of patients treated with TPN would go on to develop cholestasis, which was an easy thing to check by monitoring the patients' bilirubin levels. About the only identified risk factor for who would develop it was how long the patient had been solely receiving TPN.
If/when they developed TPN cholestasis there were a few options depending on the patient circumstance. In some cases you could decrease the amount of calories/day that the patient was receiving from the lipid emulsion and their cholestasis would improve. In other cases you could stop giving them the lipid emulsion component of TPN (TPN is a very complex product that has a fascinating history, it includes carbohydrates, proteins, trace minerals, vitamins – everything you need to grow and survive) for a period of days-to-weeks and then reintroduce it and often times their cholestasis would not immediately recur. In other cases it would, which usually meant that a pile of paperwork got filled out and even back in 2013-2014 you could get them compassionate use of Omegaven, which the FDA acknowledged seemed not to cause these problems. While the Omegaven saga was playing out, in fact another product was undergoing active study and trials in the US what was an option for patients to be enrolled in initially, or to also get via compassionate use – that product was SMOF lipid, which is used much more widely today than Omegaven is, and getting patients who'd failed the traditional soy lipid formulation onto it was the same paperwork but reliably effective.
The fact of the matter is that even today, in 2021 with fully approved intralipid (the old "problematic" lipid formulation, Omegaven, and SMOF-lipid) many hospitals in many circumstances still use the original intralipid formulation because it is not uniformly problematic, the problems are easy to detect in the initial stages (and they're reversible!), and the alternatives are significantly more expensive.
I'm an enormous critic of the FDA and many of the ways that it undermines appropriate research on a variety of things by giving out too many compassionate use waivers and wastes valuable time on time sensitive investigations, but the story of intravenous lipid preparations is not one of FDA failure. There were many failings of the wider medical establishment in recognizing and propelling Omegaven forward with more alacrity, but that is a totally different problem of insularity and arrogance.
Reading this made me realize I really don't know anything about how medical studies and regulations around them work. Why was it never a relevant question to go "Hey, do the babies over in Europe who receive Omegaven get PNALD at a lower rate?" Shouldn't that be significant enough to get you out of the "rut" of "drug is too good to make a control group justifiable, but doesn't have enough evidence behind it for people to care."
I have some understanding that the FDA prefers (or requires?) American studies, or something, so maybe this is the problem?
I'm pretty sympathetic to the need for "cowboys" in medicine, because those are the kinds of doctors who actually got transition medical care for trans people to the semi-tolerable point where it is today in some countries. It took some surprising guts and a certain amount of generational change for doctors to actually start listening to what their trans patients were saying. And improvements in HRT protocols are all done pretty "cowboy" because of the difficulties of getting any decent study funded and all the drugs involved being used off-label. The official guidelines from groups like WPATH or my local org Rainbow Health Ontario, while less bad than what was done two or three decades ago, are still pretty suboptimal in a lot of ways based on some of the preliminary stuff that comes out of clinics which are willing to experiment a bit. If you get progesterone as a trans woman (which seems to help complete breast development, improve libido, etc.), you have to sign a waiver which says "officially this does nothing, but we agree to prescribe it to you anyway", and many doctors just won't prescribe it because officially it does nothing. There's also a bunch of misinformation about the risks because the official guidance doesn't distinguish between different types of progesterone medications, some of which have significantly worse side effects than others. Repeat for a variety of other medications and combinations thereof. You need someone willing to do their own research and push the boundaries in order to get the best care.
At the same time, it's a bit of a double-edged sword. One such cowboy in the UK was trying experimental psychotherapy of dubious validity (specifically, therapy intended to make a gender-dysphoric person not transition, which has little evidence to support its effectiveness and a good amount of evidence to show that it actually causes harm, as I hope you know) on non-consenting children who called a mental health helpline about their gender troubles.
This is also why I have some fearful uncertainty about some of your suggested changes to FDA approval and drug coverage in a previous post. Because transition medications and other procedures will basically never get a good randomized controlled trial done for it for a variety of reasons, it would be hard for it to get the kind of full approval needed to force insurance coverage in your suggested system, even with cost barriers to getting certification removed, which leads to a big mess of coverage or lack thereof (coverage is already a big mess, I foresee this making it worse) and it being used by red states to score culture war points, as trans care for minors currently is.
Am I the only person who finds the statement 'Every single one of the parents of a child in the control group noticed such inferior results that they begged and pleaded with us to let them have the treatment and we being softies gave it to them so our results are weak' more than a little bit insane? Like, surely whatever early data the parents were noticing was so strong that you could have recorded it and used that as the basis for robust results? Then again, there is the catch-22 of not letting anyone do a study to prove that it's effective because it's too effective for it to be ethical to have a control group, so there's plenty of insanity to go around here.
I'm amazed that nobody has mentioned the controlled trials on the use of oxygen in premature babies. It's my understanding that the doctor who did the study knew that he was going to blind some of his subjects and possibly kill a lot of them, but that has prevented a lot of babies from being blinded since then. I don't know what it's ethical to do, but in cases where the accepted treatment is worse than doing nothing and it's almost universally used, one can make a strong case for acting.
Oh and apropos of nothing, THANK YOU for doing psychiatry. I keep thinking about how lovely that is. <3
I appreciate the deep-dive post. I actually came away from this thinking that it somewhat weakens the overall argument that the FDA is in the “wrong” in this case.
> I realize the hurdle is there for a reason.
I’d be really interested to see you flesh this out further.
I think we are covering a subject where it’s just not valid to look only at the costs of the current regime. We need to also make an effort to compare the benefits too. A zero-regulation environment (or even just a move directionally towards less regulation / faster approval) would clearly also involve harms, or put differently, the status quo is averting real harms. The question is whether for a new position on the spectrum, the new benefits would outweigh the new harms.
Thalidomide is the obvious one that the FDA remembers deeply, but I think it would be more interesting to (say) investigate the covid vaccine candidates that didn’t get through the approval pipeline, or some other drugs that were dropped at phase 3/4 due to side effects that were only detectable with very large studies. Essentially, try a steelman of the FDA’s current level of conservativity based on the public record.
Another related critique I’d make of this piece is that the drug approval process is complex enough that I’d be very suspicious of claims that you can make a localized change to fix one small scenario like “allow Omegaven to be used when it clearly works”, without having substantial knock-on effects on potential widely-used drugs that would swamp the positive utility of fixing this case. For example, would we also get statins which are more efficacious but we later discover have bad side effects? My impression from conversations with people in the field (so a low-confidence assertion) is that there are lots of drugs where the cost/benefit ratios are not that far below 1, and so undetected side effects would take it from a net benefit to a net harm. Would be interested in others’ thoughts/input on this angle. To be clear it does seem to me that there are improvements to be had, but I think we’re optimizing a U-shaped function and need to be careful not to make things worse.
Huge fan, but I think Scott's being too easy on himself here. This was THE example that the rest of his initial FDA takedown hung on - the implication was "throw a dart at the FDA and here's what you'll find, it represents a thousand other stories just like it." It turned out to be substantially wrong. Like, AT LEAST as factually wrong than the crappy NYT story about Scott which he (correctly) didn't excuse.
Scott's charging for content now, and is doing important, high profile investigatory journalism. It's not OK to get so many relevant facts wrong. "This story is directionally and emotionally accurate, so don't worry that my supporting arguments were wrong" is exactly the kind of BS I want to see ACT/SSC as fighting against in the rest of media.
I would've liked to see a full retraction here, and most importantly, I'd like to see some of the subscriber money go to an intern who can help fact-check.
(and again, I'm a huge fan -- that I can even imagine Scott taking the high road here is the reason this disappoints me so much more than the typical BS in another publication.)
FDA has recently simplified the process for applying for "Expanded Access" to prescribe non-approved medications. FDA says they approve over 99% of applications. Maybe it is easier now compared to when you tried? https://www.fda.gov/news-events/expanded-access/expanded-access-information-physicians
Has anyone done a randomized controlled study to prove that the FDA is both safe AND effective?
If no one has done such a thing, why are we using it?
That omega-6-based nutrition causes liver issues in newborns isn't surprising, given that humans evolved consuming extremely low amounts of it.
https://www.jeffnobbs.com/posts/why-is-vegetable-oil-unhealthy
I cannot help but think of a Dr. Gura championing a fish oil treatment as having the voice of a certain shark vTuber.
> What would a still-default-no-but-the-hurdle-is-smaller medical world look like?
How about automatically approving any drug or treatment approved by at least N other first world regulatory bodies, where N is ideally equal to 1.
Why couldn't they compare death rates between the USA and Europe, where Omegaven was already being used? Too many confounders?