I think the especially weird thing is that in the historical analogy of AZT for AIDS, after the trial was stopped early, the FDA issued thousands of Compassionate Use Exemptions so that people could use the drug even while it was getting approved.
One possible (but still IMO unlikely) reason delay isn't so bad: logistics of getting high risk people tested within 3 days of symptoms and getting them the treatment, are harder than we think, so Merck and/or healthcare providers are working on a way to roll that out easily.
If the trials were done on people within 3 days of showing symptoms, how well does it work on people who take it at longer intervals?
Is it a case that "works great if you get diagnosed within 3 days, isn't better than aspirin if you get diagnosed 8 days"? Questions like this are what interest me about it.
My understanding so far was, that you need to give it early, in the first days after the onset of the symptoms.
Which is obviously an issue, because most people have lighter symptoms for a couple of days up to a week and develop worse symptoms only afterwards. If this holds true, you can't give the medication specifically to those persons in risk of complications or death, because you don't know early enough who those are. Except general risk factors of course. I remember deliberations on whether to give medication like this as a precautionary measure to everybody with a positive test result in future outbreaks in retirement homes this winter (context is Germany). Or to everybody with symptoms who cannot be vaccinated. I don't remember if they were talking about Paxlovid or sth. else, could check that. There shouldn't be that many options I assume. The underlying assumption of course is: even if there'll be an effective drug this winter, there'll not be enough of it to give it to everybody. It would still make a difference.
There might be a weird backwards sort of logic here where, because the population with AIDS was so heavily stigmatized already, people were more willing to break normal medical ethics rules for them. I can imagine a scenario where more stick-in-the-mud types can't be bothered to care, and more forward-thinking folks prevail. Meanwhile with covid, the stick-in-the-mud folks who never gave a shit about AIDS patients suddenly need to grind everything to halt in case one random boomer has a bad interaction.
I think the appropriate term is "sympathetic", not "stigmatized". In the 1980s, homosexuals were a sympathetic population to the sort of people who e.g. run the FDA. Here and now, the people with COVID are stigmatized - they're a bunch damn dirty antivaxxers, in some jurisdictions excluded from normal social and economic life and/or forced to wear a de facto batch of shame. But they're not at all sympathetic to the Blue Tribe elite.
True, not every COVID patient is unvaccinated. But neither was every AIDS patient gay and/or an IV drug user in the 1980s. And when it served the narrative, we'd highlight the people who got AIDS through e.g. blood transfusions then, just as we highlight the breakthrough infections now. But only so long as it served the narrative, then everyone pivoted right back to AIDS as a plague on the sympathetic gay community, or COVID as a disease of the stupid antivaxxers.
Dr. Marty Makary, who has really become my hero, has an Oct 7th op-ed in The Wall Street Journal arguing the FDA should allow expanded access to Molnupiravir
"the FDA could allow compassionate use of molnupiravir now by activating what is known as an expanded-access protocol, ahead of the drug’s formal evaluation. There is a precedent. Last year, the FDA activated the protocol for convalescent plasma months before it issued a formal EUA. The agency also allowed monoclonal antibodies to be given to President Trump and other officials last year before that therapy was granted a formal EUA."
he also claims:
"Molnupiravir pills are ready to go. The government has prepurchased 1.7 million treatment courses. The medication doesn’t require refrigeration and is easy to ship. These pills are sitting on a shelf as Americans are dying in hospitals."
Obviously Makary's arguments extend to Paxlovid too.
The manufacturing argument can explain why the FDA wants some more time to check if the mass produced drug is as safe as the trial drug. But it doesn't explain why they would then stop the study. Presumably, Pfizer has already produced enough to finish the study. For the people in the study, having a 50% chance of getting life-saving treatment is unambiguously better than a 0% chance. And we get more data.
I could be wrong but I believe stopping the study means the placebo patients are switched to the real drug - that's what stopping it for efficacy means.
That has always been my understanding of what happens when trials are stopped because the substance is too efficacious - the placebos are all given the real thing too, they aren't just released into the wild.
I don't have any experience but I did find this article on the ethics of truncating trials from 2007 which implies that this is not standard practice (or was not, in 2007):
> "Finally, stopping a trial early does not guarantee that current and potential trial participants will receive the apparently beneficial treatment (assuming that one believes they should). Studies of dissemination of new treatments reveal that long delays, such as those between reports of randomized trials and recommendations of experts in review articles and textbooks, are common (14). Continuing a 2-group trial gives participants at least a 50% chance of receiving the experimental treatment, whereas if the trial is stopped early, the probability that participants will receive the treatment due to rapid dissemination is likely to be considerably less than 50%."
I think I've been imagining treatments for a chronic condition, where all participants are enrolled at the beginning, and half get one treatment and half get the other. In that case, they usually give the placebo group the treatment. But I suppose with the covid trials, they are constantly enrolling new patients and randomizing them to one half or the other, and if they stop, then they stop enrolling new patients.
No. When designing a trial, you have to make a guess as to the lowest size effect you expect to see. You then have to design the size (number of patients and duration) of the trial to give you a pre-determined chance of detecting that size effect. The smaller the size of the effect, the larger the trial has to be to reliably detect that effect. If you're lucky and the effect of your drug is substantially greater than the effect you designed for, then your trial will reach your pre-determined significance well before you get to your original trial size. The statisticians monitoring your trial will notice that and say "Hey, you met your significance goal. It's a success. You can stop now", and if the FDA agrees, you stop (because money and there's lots of other things to spend it on to get this drug approved). There's nothing so far about giving treatment to the placebo arm. For some trials, you enroll a bunch of sick people and start giving them drugs, or placebo. If the treatment arm suddenly rises from their wheel chairs and casts off their crutches, the statisticians will give you the nod. But now what do you do with all the sick placebo patients? That's really up to you, but drug companies don't want to be seen as evil, so they will often just give drugs to the placebo arm. I don't think this is an FDA requirement, nor do I think there is some pre-determined criteria for doing that. In the present case, because they wanted to give the drug in a fairly short window after symptoms appear, the patients are enrolled as they present themselves to the trial, not in a big bolus at the beginning. As it appears to be the case in this trial, they got the nod from the stats at some point, but there wasn't some ward with whole bunches of sick people lying around with placebos that you could now dose with the good stuff. Since it's a rolling enrollment, the people who were going to get placebos haven't shown up, yet. They aren't sick yet. So, you just close the doors and say the trial's over. There's no need to keep enrolling people into a finished trial, hence no more people to get placebos, or offer treatment. There may be a few people who arrived in the short window just before the stats called time that might be offered treatment just because they're already in the door. I don't know.
TL;DR Stopping for efficacy happens because you proved it works, not because you have to switch placebo patients to treatment.
But that's not what Scott's original post said. Apparently, trial was stopped early because "it would be unethical to continue denying Paxlovid to the control group". I guess that declaring this doesn't technically mean that the control group would in fact receive treatment, but the alternative seems just a bit too kafkaesque to be true. inb4 being shown otherwise, I guess...
Who said "trial was stopped early because "it would be unethical to continue denying Paxlovid to the control group"???? The press release, which is the only official communication I have see on the subject, says "At the recommendation of an independent Data Monitoring Committee and in consultation with the U.S. Food and Drug Administration (FDA), Pfizer will cease further enrollment into the study due to the overwhelming efficacy demonstrated in these results and plans to submit the data as part of its ongoing rolling submission to the U.S. FDA for Emergency Use Authorization (EUA) as soon as possible." There is >nothing< in that statement about ethics or giving anything to the control group. Scott's original post may have said that, but having read the references in Scott's original post, there's nothing authoritative that says anything like that. Scott does point to a Tweet: https://twitter.com/KelseyTuoc/status/1461781455407828993 that claims "they'd stopped enrollment into their Paxlovid study because it prevented 89% of Covid deaths, making it unethical to not give it to controls.", but there's no reference given to back that up. So, my take is that Scott just blindly repeated some rando's uninformed tweet and everyone who thinks Scott is a careful scholar of drug trials has taken it as gospel. Scott also says "Perhaps there’s not enough evidence for the FDA to be sure Paxlovid works yet? But then why did they agree to stop the trial that was gathering the evidence?" As I've pointed out in other comments, Pfizer has 10 other human trials, 7 ongoing, that are still gathering evidence. So, in fact, Pfizer has not stopped all trials gathering evidence, a fact that Scott, or anyone else, could have found in 5 minutes by looking at clinicaltrials.gov.
I love internet comments who know what they are talking about.
There is also an unfortunate tendency for people who are experts in one thing to assume that the experts in another thing are idiots. A random example might be a consumer electronics engineer talking s_it about automotive electronics engineers because they still do X. And the automotive engineer might say, "Yes, we could do it your way. If we were designing cars to be used indoors. We need products that work as well when parked outside all night in Alberta in January all they do when parked outside all day in Dubai in August." Ohhhhhhh....
> the automotive engineer might say, "Yes, we could do it your way. If we were designing cars to be used indoors. We need products that work as well when parked outside all night in Alberta in January all they do when parked outside all day in Dubai in August."
Do they? I would estimate the number of people interested in that use case at somewhere between zero and one. It's just not that common to move cars overseas.
The manufacturer might get economies of scale by producing the same cars for Alberta that they do for Dubai, but that's not a business need (there are plenty of businesses that sell more than one product!), and from what I've read, it's also pretty unlikely that a car legal in one country will meet the legal requirements of a second country.
I think you'll find that the vast majority of high-volume cars outside the US market are very much designed to be sold and operate in multiple countries. If I buy a BMW in Australia, it will meet the Euro emissions standards, and it will not be significantly different to the same model sold in Sweden.
Individual consumers may never take it between the two extremes. But every car on the lot could end up in either extreme and consumers don't want to care to pick only certain kinds off the lot.
Even if we just focus on the continental U.S. market, changing the original statement to "Minneapolis in January" and "Phoenix in August" makes the same point.
I'm not sure if I buy the mass-production-check reasons given.
To be a big drug manufacturer, it seems like you must already have very robust post-production quality control processes in place, and these are going to look more or less the same for any drug. For each batch, pull X number of random samples, check that they look like the batch from the trial and that variances are sufficiently tight, etc. This process, while not foolproof, seems like it should at least easily be strict enough so that some extra perusing from the FDA will have no marginal utility. That is, if the problem is subtle enough to pass the pre-existing post-production QA check, it will be too subtle for the FDA to detect. (And of course the FDA could take random samples in real-time and run it through their own lab.)
Is anyone aware of a historical case where the drug company said "Hey, we're ready to go" and the FDA came along and said "Doh! Vat 76 in production line 4 has too much temperature variance"? I'm skeptical.
With regards to how well the manufactured drug maintains its efficacy in storage, it seems like a provisional approval could just include a requirement that the initial expiration date is no further away from manufacturing than those used in the trial.
1) Having worked in a production environment (though certainly not anything pharm related), I bet something like that happens semi-regularly (either issues are found during the FDA review, or issues are found by the company while preparing for the FDA review which is just as good).
2) If the FDA isn't doing these reviews, the "existing" QA checks probably don't exist, or at least wouldn't be nearly as thorough.
To take my personal anecdote as an example as well, I work at a black box trading firm. Obviously, we have tons of checks at many levels to prevent bad strategies from going to market. Are they foolproof? Certainly not. But I'm pretty confident that there's no way that some outside group of even really smart people could come in and find a bug without basically becoming a full-time employee, and even then it would be very hard because, obviously, the code was written by tons of smart people who are extremely worried about a bad strategy hitting the market.
I do think an outside group could analyze our outputs and find signs of a problem, if not the exact cause. It wouldn't be easy, but you could look for clearly infelicitous patterns and whatnot. (Obviously, we do a lot of this ourselves, too.)
Similarly, I think any value-added from the FDA on QA would have to come from analyzing the outputs of the manufacturing process, i.e., the actual drugs, which I think could be done concurrently with approval.
As an example, back in 2015 the Swiss Frank made a 58 sigma (?) move against the Euro and various people lost their shirts. The issue was the math said a move of X should only occur once in 200 billion years or whatever. But obviously that had nothing to do with the Swiss National Bank's decision to intervene in the currency markets.
I can totally see an organisation falling into group think because some core part of their model has worked so well in the past.
Oh, sure. But no outside group would have ever gone in and been able to assess that any better. The outside group would be just as subject to group think and had lower incentive to quibble over the math.
I don't think the idea is that regulators would look for bugs directly. Rather, the regulator makes sure *you* are looking for bugs, by requiring evidence in the form of documented processes and excessively long reports.
I'm not sure if hedge funds have the same requirements, but AFAIUI this is basically how it works at banks.
In your case, the incentives might line up well that you do lots of QA. But that's not always the case, and that's where there's opportunity for value-add from regulators.
The results on Boeing's QA after they were allowed to do the regulatory parts in-house suggests that having a regulatory set of checks, while not necessarily directly useful, may incentive-wise be a necessary evil nonetheless.
> Is anyone aware of a historical case where the drug company said "Hey, we're ready to go"
> and the FDA came along and said "Doh! Vat 76 in production line 4 has too much
> temperature variance"?
There are FDA warning letters that say: "On inspection, we found pigeons were roosting in the cleanroom in your facility" or the like, but those aren't relevant here. Instead they will ask "How hot can Vat 76 get before the batch is ruined, and you have to tip it down the sink?" Possible answers are, in order of increasing acceptability:
-"I dunno, we always do it at that exact temperature, and nothing bad ever happens"
-"I dont know the exact temperature where it goes bad, but this one time when it was 50 degrees over, the batch went crispy on the bottom like a paella"
-"We tried it at a range of temperatures, and at 10 degrees too hot you start to start to see a new impurity forming"
-"We modeled the interaction of temperature, concentration, and pH, and we can show you that any combination of these that lie within this 3D Design Space will give you an acceptable product"
If all your answers are like the last one, your manufacturing process will be approved. Anyone can notice if something went outside spec during manufacture; the FDA want you to justify why you chose that spec in the first place, so you can prove that your choice of temperature control methods was correct.
How often does this matter for new drug approvals? The best answer I have is this paper:
"Scientific and Regulatory Reasons for Delay and Denial of FDA Approval of Initial Applications for New Drugs, 2000-2012"
302 new drug applications were examined, of which half were approved immediately, and another quarter of the applications had "deficiencies" to be resubmitted before ultimate approval.
"We limited our analysis of [manufacturing] and labeling deficiencies to
products that were not approved despite satisfactory safety and
efficacy (n = 23). Chemistry, manufacturing, and controls deficiencies
included problems with dissolution or manufacturing
specifications, incomplete stability data, high endotoxin
levels, and deficiencies noted during inspection of manufacturing
facilities."
Note "endotoxin" is a component of bacterial cell walls. Its presence in your drug, besides leading to unwanted immune reactions in patients, indicates that at some stage you left the batch sitting around long enough that bugs started growing in it.
Results: 17 drugs of 23 failed the first cycle review due to chemistry, manufacturing and controls issues, two for CMC plus labelling; the other 4 were arguing about the labelling alone. However, most of these were eventually approved (i.e. the problems were fixable). Only 4 were never approved, and I would bet some or all of these were just because the pharma company ran out of money/interest and stopped trying.
Regarding stability, we saw the impact of this with the emergency approval of the Pfizer vaccine. Remember how at first it needed to be stored at -80 celsius, leading to supply chain issues all over? But then after a while Pfizer announced it was ok at higher temperatures (I forget the details) and you could leave a vial thawed at room temp for a little longer before use. That shows that Pfizer didnt have all the stability data ready, but the FDA let them go ahead anyway, as long as they were super conservative with their choice of storage temperature/shelf life.
> To be a big drug manufacturer, it seems like you must already have very robust post-
> production quality control processes in place, and these are going to look more or less
> the same for any drug.
I'll blame TV for this one, where the hot young chemist squirts a sample of drug into a machine, then walks round to the printer and rips off a printout of everything thats in the drug sample.
In reality, with a new product that noone has ever made before, the chemical properties of the product itself and the impurities and degradants is unknown. With a known product you can buy a certified standard that says purity >99% on the label, and a list of whats in the other 1%, and compare your own material to that. If the compound is brand new, you have to invent the purity test yourself, show that it works reproducibly enough that you can certify the results are true, and justify that process to the FDA. Then guess what the possible impurities are, make or buy samples of all those to compare, and invent robust tests for those too. Then figure out the chemistry of how the impurities got there, how to purge them, and how to prove they are gone from the final product. The whole time you could have got it wrong, and there was something hiding in your original "pure" sample that you never found. I'll illustrate this with more on the ranitidine story I mentioned earlier (apologies if I get any details wrong, I've never worked on this myself, only read about it in the news).
Ranitidine (Zantac) has been used since 1981, and was the world's biggest-selling prescription drug by 1987. It was considered extremely safe and people took it daily for decades. Countless manufacturers must have made batches of this stuff over the years, but noone discovered until 2019 that if you keep the stuff lying around, it possibly decomposes into something carcinogenic. This is the chemistry of the drug itself, not due to manufacturing problems as far as is known, but its still an unsolved problem. See FDA website: https://www.fda.gov/drugs/drug-safety-and-availability/questions-and-answers-ndma-impurities-ranitidine-commonly-known-zantac
"Q. How long has NDMA been in ranitidine?
A. FDA does not have scientific evidence to determine how long NDMA has been present in ranitidine products."
How did this happen, with all the quality control and testing of every batch ever made? The answer is, there was no test for NMDA so noone tested for it, the levels are small enough that it was hidden, and all the batches were certified "pure" and safe. It was only discovered by chance, through a different drug called valsartan. Clever manufacturers figured out a new way to make valsartan that was much cheaper than the old process, thus giving an edge in the highly competitive generics industry. The new process used sodium nitrite, which is well known to form nasty carcinogens e.g. by reacting with amines naturally found in meats etc. The valsartan manufacturers knew this, but since their process had no amines in it, they knew it was safe, and were easily able to justify this new process to the FDA. The problem came on scale-up, when recycled solvents were used, that came from a process where amines *were* used. The solvents were purified, but traces of amines remained, and they didnt test for them or control them adequately. This meant batches of valsartan were released that had relatively huge quantities of NDMA in them, enough that they behaved weirdly, and so someone investigated after the fact. (Insert quote here from Asimov: The most exciting phrase to hear in science, the one that heralds new discoveries, is not “Eureka” but “That's funny...”). The valsartan batches were quickly recalled, a new test for NDMA was invented, and valsartan users were safe again. But of course, now that testing for NDMA was routine, people wondered what other drugs might be contaminated, and the rest is history. The problem with ranitidine is that the required amine is present in the molecule itself, along with the nitrite, and the FDA is worried enough about conditions where the two might meet, that the product has been recalled until this is sorted out.
> In reality, with a new product that noone has ever made before, the chemical properties of the product itself and the impurities and degradants is unknown.
My first thought was that mass spectrometry might be an "easy mode" first pass check for impurities and degredants, as it would highlight anything in the sample that doesn't have the same molecular mass as the things you expect, so you could follow up with chemical tests to see what those things are with the candidate space significantly narrowed by their known mass.
The I realized there were at least a couple huge problems with this. The first being that starches are very common intended inactive ingredients in pills, and starch chains can be any multiple of the constituent sugar's molecular mass.
The bigger problem is going to be isomers and related molecules of near-identical mass. These seem like they'd be very common impurities in manufacturing an organic molecule, and they'd be hard or impossible to differentiate from the intended product on a mass spectrometer.
- HPLC-MS: this is a chromatography column hooked up to a mass-spec machine. If you're flush, you do a fragmentation mass spec, where you get both the mass of the original molecule and then hit it with electrons to break it into pieces and get the masses of those as well. This works very well if you know what you're looking for.
- NMR: nuclear magnetic resonance. This is how you figure out what is present if you don't know what it is. If you have a common contaminant, you'll purify it with some kind of chromatography, then run an NMR on it. The magnetic resonances of the hydrogens (or occasionally carbons) give you very detailed chemical information.
So if you know what the desired product is, HPLC-MS is an easy way to measure how much of what you want is there. If you have contaminants, NMR is the common way to figure out what byproducts you're getting, and then you can figure out how much of a problem that is.
Related molecules of identical mass are not much of a problem with the chromatography. Even isomers will get caught out with fragmentation mass spec pretty easily. You can occasionally have problems with racemic mixtures (e.g., molecules that differ from each other at only a single chiral center). Racemic mixing is a problem, for example, in methamphetamine manufacturing.
Even there you can do, e.g., circular dichroism spectroscopy to evaluate it. (Look at how polarized light rotates through the solution, which depends on the chiral centers).
This seems like an unusually fast turnaround time for highlights-from-the-comments. Or maybe it just seems that way because we got two such posts in short succession and I don't remember how long ago the Ivermectin post was.
There is indeed real urgency to this situation. Unfortunately I don't think any of us have a good theory of change as to how to actually change things, but screaming into the wind is still better than nothing.
This whole thread seems to be predicated on the misconception, from Scott on down, that this efficacy trial ( NCT04960202) is THE trial that will determine if the FDA can authorize / approve it. That is simply not true. There are 10 more, 7 ongoing, human trials registered at clinicaltrials.gov, each designed to test a different aspect of efficacy and (mostly) safety. NCT04960202 was the big efficacy trial, and passing that is a huge milestone in the approval process. (Sometimes you can rescue a efficacious drug that has some bad safety signals if you have a clear understanding of the harmful mechanisms, but I've never heard of of a non-efficacious drug that was brought back to life.) But, safety is still a big hurdle in the approval process. In addition to the registered human trials, there are dozens, if not hundreds, of animal studies, that you and I will never see, that also go into the FDA decision process. Nobody here has any idea what the results of those are. (OK, we know that not all the rats died, or the human trials would not have taken place, but there's a lot of room between there and "its as harmless as distilled water, or rainwater, and only pure-grain alcohol".) The results of all those studies runs to hundreds of thousands of pages, and anyone who claims FDA can go through all that in 24-48 hours without cutting corners is talking nonsense. Basically, the press release that comes after a good efficacy trial doesn't tell you much about the real state of the approval process, or when you should start grousing about the FDA dragging it's heels.
The fact that the efficacy trial was halted merely indicates that FDA is satisfied that you checked that box in the approval process and that Pfizer can put its resources to checking the other 99 boxes it takes to manufacture, market, and distribute a new drug to patients. To halt a trial for futility means FDA will never approve this drug no matter how well the rest of the trial goes and you should just stop putting people at risk.
This is a good thought, but if this is correct, I would no longer think that BronxZooCobra's above reply to TheChaostician has force. If the FDA is still reasonably worried about safety, that I think it's hard for them to reasonably say "we already know there is such a huge, positive expected value that it would be unethical to give a placebo any longer so we have to stop the study".
FDA didn't force anything. "At the recommendation of an independent Data Monitoring Committee and in consultation with the U.S. Food and Drug Administration (FDA), Pfizer will cease further enrollment into the study due to the overwhelming efficacy..." Basically, FDA concurred with Pfizer that they showed efficacy and they don't need to do any more to convince FDA that it works. That actually can speed up approval since nobody's waiting around for the results when the outcome is already known. I don't think halting the trial has anything to do with the patient's well-being. But, there are still more safety trials to complete, so it's not fair to say that "we already know there is such a huge, positive expected value" because the other side of the "expected value" calculation, i.e. safety hasn't been determined, yet (Maybe. I don't really know what that state of approval really is, but nobody else here knows, either.)
Yeah, but regardless of who forced what, there's still an inherent contradiction, IMO.
If what BronxZooCobra said above is correct, stopping the trial means they start giving the placebo arm participants the real drug. Well, if it's OK to give the placebo arm participants the drug, why isn't it OK to give everyone else the drug? Obviously, trial participants may systematically differ from the general population in terms of comorbidities, other medication, etc., but if so, it seems you should just go with Scott's idea of a phased approval where you only give it to members of the general population who satisfy the trial inclusion criteria.
"The results of all those studies runs to hundreds of thousands of pages, and anyone who claims FDA can go through all that in 24-48 hours without cutting corners is talking nonsense"
This just raises more questions:
1. How many of those pages contain information that's actually important?
2. How often does the FDA spot problems in one of those hundreds of thousands of pages? How often are those problems serious?
3. How many people did the FDA commit to going through those documents? Why not more?
4. Is the FDA working 24/7 on this approval given the number of people that could be saved or are they taking time off? If they are taking time off rather than working in shifts, why is this the case?
5. Where's the cost-benefit analysis on all of this given that this drug is primarily designed for high-risk unvaccinated patients who have a 10-50% chance of getting hospitalized otherwise and a 5-30% chance of dying?
6. How much of the work could've been done by the FDA in advance rather than waiting for the trial to be over?
7. How much of the work is repetitive and could be parallelized or skipped altogether for someone as bigger as Pfizer?
8. What's the SLA at each layer of interactions between Pfizer and the FDA and how much do communication latencies affect the time required to grant approval?
9. How many pills could Pfizer churn out if the FDA approved it right now? Are we actually waiting for FDA or are they still faster than the manufacturing process?
10. Why isn't approval of drugs separated into approval of an "ideal substance" and approval of the manufacturing site?
My experience with regulatory processes is limited, but nonzero. My general observation is that every part of the process has some justification (e.g. something that has actually gone wrong before), but putting it all together, it becomes ridiculously overbearing. Regulatory processes are trying to guard against human error, but in doing so, lose all the efficiencies of human judgment.
I'm not sure I see a way around it, though. When something bad happens, everyone demands "there should be a process to ensure that can't happen again!", it seems reasonable, and so yet another item is added to the checklist.
To give a non-answer for #1, if you're aiming for multiple 9's of safety, you are necessarily going to have to check an awful lot of unimportant details. I both expect and would hope that the reports have lots of unimportant details discussed at length.
To give a non-answer for #2, it's not just the errors you spot, it's the errors you avoided in the first place. Having a strict process means corners won't be cut in the first place, because they know they can't get away with it.
I seem to remember a post back in the SSC days where Scott complained that the FDA is loathe to rescind approval of a drug once it is given. That even fast-tracked drugs, approved provisionally based on surrogate endpoints, don't get pulled from the market after not doing the appropriate follow-up studies based on hard endpoints.
What's the probability that an emergency authorization of this drug will be approved, save some lives in the short-term, and then persist in medical practice for years before its use finally fizzles out? Will we save more lives on the near side of the distribution than we risk in the perpetual long tail?
And remember that trials stopped early on ethical concerns for the placebo arm consistently overstate efficacy - which is exactly what we would expect from the statistics. Not saying the efficacy isn't there, just that we should presume it's overstated based on the declared early end of the trial. Which takes a little from the sails of the argument for rushing it through a few weeks early.
Scott complains about the system we have, with some merit, but approval in the current system is what it is. A better system would allow tiered approvals like Scott suggests, but we're talking about approval the current system. Is there evidence of a strong - functioning - corrective mechanism post-approval? Or if this is a mistake, do we just have to 'live with it' from now on?
> Also, I am a bit surprised that Scott seems to use "expected value" so freely in this context. Medical ethics does not seem to work that way at all? At least, there seems to be a heavy asymmetry between action and non-action, maybe that is reflected in the approval system as well?)
"... after having identified and establishing control of all critical sources of variability, conformance batches are prepared to demonstrate that under normal conditions and operating parameters, the process results in the production of an acceptable product. Successful completion of the initial conformance batches would normally be expected before commercial distribution begins, but some possible exceptions are described in the CPG. For example, although the CPG does not specifically mention concurrent validation for an API in short supply, the Agency would consider the use of concurrent validation when it is necessary to address a true short-supply situation ..."
An Emergency Use Authorisation would be another exception. If you can put your hand on your heart and say "I believe these batches will be ok, based on this data I have" then the FDA will listen. They will review your data for themselves though, before giving you the go-ahead, and that is probably whats going on right now for Paxlovid.
Dr. Marty Mackary [sic] says this : "As a Johns Hopkins scientist who has conducted more than 100 clinical studies and reviewed thousands more from the scientific community at large, I can assure you that the agency’s review can be done within 24 to 48 hours without cutting any corners."
"
I remember reading this when it came out, and I'm not sure it's credible. I suspect the 24-48 hours is an estimate for an academic review, not a regulatory review. I don't have any knowledge of the FDA's internal processes, but normally a regulator can't just say "looks good to me", they have to file hundreds of pages of reports demonstrating that every item was scrutinized and considered.
Everyone involved *should* be able to write and/or review a draft of all those reports well in advance, with everything but the final numbers and e.g. the case reports of the people who develop significant adverse reactions in the last week of the trial. And it should be possible to make an approval decision contingent on the final numbers coming within a specified limit. We do that routinely in e.g. space launches.
Filling in the final blanks and making sure they are in range, should be a 24-48 hour task if you need it to be. But it wouldn't surprise me if parts of the bureaucracy would say "That's not how we do things around here, so NO".
In the short term, the pills may do more harm than good: if we start with a limited supply, it would make sense to prioritize the pills for the unvaccinated, since they are much more likely to have severe outcomes. If this becomes a part of the pill allocation process, it would look super awkward, as if we are punishing the vaccinated for doing the right thing. If some people are on the fence about getting the vaccine, knowing that getting vaccinated might hurt their chances of getting the pill might push people towards not getting vaccinated, and if this happens to a significant degree it could outweigh the benefits of a limited supply of the pills.
Read a piece published today in Stat about challenges and obstacles anticipated in deploying Paxlovid and molnupiravir, including remaining questions about how diagnosis may impact when and whether they can be prescribed: https://www.statnews.com/2021/11/23/covid-antivirals-pandemic-game-changers-americans-struggle-access/. It sounded like answering these questions from a regulatory standpoint might be a major contributor to the time needed for the FDA to approve Paxlovid - because it sounds like they need to issue guidance on whether a PCR positive Covid test is needed, or for which populations it may not be, or if it can be prescribed later than three days post-symptom onset (because the study only covered how well it works when given within three days of symptom onset). One person quoted suggested that the drug(s) be authorized for prescription some immunocompromised people preemptively in response to Covid exposure without a positive diagnostic test. But with only the data available from the studies at the time of the FDA meeting, it sounds like the FDA needs to be able to issue guidance on all these things, many of which which are not directly answered by study data alone (and ideally make the guidance mostly feasible for real-world conditions where e.g. Covid testing turnaround time may not be very fast). Am I correct in assuming this is a major contributor to the time to approval in this process? These seem to pose a real challenge (given real-world limitations in distribution of and access to the drug, as well as safety considerations - even if you had unlimited supply, the cost/benefit ratio for preventative prescription of Paxlovid for Covid exposure or suspected Covid might not pan out in all populations just because post-diagnosis treatment does) that consumes time for reasons other than bureaucracy just takes time because it has to take time even in emergencies.
This is exactly what makes me so mad about arguing on these issues. The same people who defend the legal regime that totally bans drugs which the FDA hasn't yet found to be sufficiently beneficial (tho often claim to believe there is a constitutional right to autonomy which lets women choose to abort babies, which I love the idea of but surely that creates some similar right to take unapproved potentially lifesaving meds) then use the worries about lack of trust to justify being extremely reluctant to approve anything.
It's a vicious bootstrapping which seems largely based in the inability to imagine that people could adjust to any other message. Sure, now we may equate FDA approval with certain features but there is no reason the FDA couldn't wake up tomorrow and tell ppl only green boxed meds are tested to that degree and (with a bit of PR) most ppl and doctors would adapt.
>Scaling up chemical manufacturing is not trivial (a regular contender for Understatement of the Year). E.g. heating and stirring work differently in different sized reactors. Heat transfer in and out of your reactor works through surface area, but heat produced/consumed by the reaction depends on volume. If your stirrer design isn't right for the viscosity of the solution, you might get hotspots and so on.
>Ideally, the FDA expects you to understand the chemistry so thoroughly that you know everything that can possibly go wrong, and design your commercial process so that none of these things can possibly happen. The commercial batches will therefore be identical *by design* to the clinical trial batches, and you have to prove this with science. Of course in practice you don't have to have collected all the evidence before you can start selling batches, but you must have your plan in place with a solid scientific justification for every decision you made along the way. It also helps to have made a statistically valid number of commercial batches to show that your beautiful process works as designed (how many batches is that? you tell me, and justify your decision).
>Pfizer/Merck will have thrown everything at this problem alongside the clinical trials, as they can afford to do this, so their regulatory submissions will be pretty good. However they still might have to store the new batches for a few months to demonstrate that they have a comparable shelf-life to the old batches, and FDA might wait to see this data etc.
I do not understand why the FDA would be concern itself with the details of the manufacturing process when they can just require a chemical analysis of a representative sample of the product.
I get that for mRNA vaccines, this may be insufficient because you also have to check the size of the lipid nanoparticles and that the mRNA is actually within them, but for the Paxlovid, Wikipedia suggests that the active ingredient PF-07321332 is just a straightforward organic molecule.
I am not a chemical engineer, but between mass spectroscopy, chemical analysis (for isomers) and in vitro testing (?), I feel that modern age chemistry might be up to the task to decide how identical two pills are two each other.
I feel the selection of a process of manufacturing should just be a business decision: giving a probability estimate of a certain process chain to fail and ruin your bunch, optimize for maximum expected gain, or something.
The same goes for the shelf life. If the pills are sufficiently identical to the trial ones, I would assume that their shelf life is similar as well. All the counterexamples I can think of feel rather constructed, like "We replaced Phosphorus 31 with P-33, now our chemical has a half-life of 25 days" (and skipped mass spectroscopy, and also the chemical costs three orders of magnitude more) or "we put H2O2 into the filler material, and given time, it reacts with the active ingredient over time" (so the pills were obviously not chemically identical).
If the product is easily chemically analyzed, it should be enough to regulate the product: When I buy table salt, I do not really care if it comes from sea salt or the salt mines or from someone burning sodium in a chlorine atmosphere.
Only if the product is a complex mixture of different chemicals regulating the process chain makes sense: a complete microbiological analysis of a representative sample of soy burgers out of ever freezer is not cost effective, so we mostly regulate the manufacturing process, cold chain, and use-by dates and sample only sporadically.
I feel standard chemical drugs (e.g. not mRNA vaccines or the like) are mostly the NaCl case, not the soy burger case.
That is not to say that upscaling the synthesis of new chemicals is easy, only that trial and error should be a valid strategy. Any new drug even moderately effective against COVID-19 stands to make billions. If the manufacturer has to throw away the first few millions of doses because the selected process did not work, this should not affect their bottom line too much beyond the added delay to market.
> I do not understand why the FDA would be concern itself with the details of the manufacturing process when they can just require a chemical analysis of a representative sample of the product.
Because then you don't know the failure modes of the manufacturing (e.g., lower efficacy might be acceptable, a poisonous byproduct would not be) and how likely they are. The FDA can't test every batch of product.
> I do not understand why the FDA would be concern itself with the details of the
> manufacturing process when they can just require a chemical analysis of a representative
> sample of the product.
Its firstly because no chemical analysis measures absolute purity, you only get a measure of something else plus an associated experimental error. Each separate test or comparison you do adds further assumptions, until the final combined results are just an approximation of the "true" purity/potency. In mass spec for example, every component ionises differently, so your quantitative work has to correct for that. Some components don't ionise well, or not at all, and if its something new that you weren't expecting it could be hidden in the noise. Other components are too light or easily fragmented or swamped by matrix or whatever. Then there's always that one test that just doesn't work very well because the compound is hard to analyse, and so its best to leave that for Alice to do on Tuesdays when shes in, since she is the only one who can get the instrument working optimally. The whole thing is subject to the same biases and flawed conclusions as anything else in science, which of course the FDA requires you guard against (e.g. the people who measure the numbers must be employed in a separate department of the company from the people who check the numbers), but its extremely difficult to argue with a determined fraudster who is only required to provide evidence that they did the required testing. Its much harder to fool yourself (in the Feynman sense) or game the system if you have to provide evidence that you understand how to make the product correctly to begin with.
Secondly, if you have a problem with quality, no amount of increased testing of the final product is going to fix that. You can't test quality into a product; you have to design it in. You're correct that trial and error is a valid way to do this, but its better for everyone if you do this systematically from the start, rather than waiting for a batch to fail. From the FDAs point of view, its a waste of resources for them to review your drug a second time, after you didn't do it properly the first time. Its dangerous if you have to recall a batch that passed all the testing, but then you figured out there was actually something wrong with it after you already sold it (see valsartan story above). Its wasteful and perhaps unethical to have to repeat some of your clinical trials, because you figured out that your commercial process was fatally flawed and you could never get your new batches quite the same as the clinical batches.
"Maybe at stake is not the reputation of the FDA, but the reputation of the approval process?"
This but cynically. Every time the FDA drags its feet part of me thinks that they're protecting the reputation of their insane bureaucratic obstructionism. If they ever let a high-profile pill through with less than six weeks of meetings and nothing terrible happened, people might start to wonder why we're paying so many bureaucrats to sit in so many meetings.
The only reason I'm not convinced this is what's going on is that it implies people pay enough attention to the FDA to notice what they're doing.
Seems plausible, except swap unspecified people for other rival bureaucrats. Each agency surely strives to present itself as doing indispensable highhly demanding work, so when the budgetary pie gets divided they would receive an appropriately large share.
Since you took a lot of trouble to come to a numerical conclusion about Ivermectin, I'm curious how certain are you that the FDA's Paxlovid approval process will cost tens of thousands of lives. Eliezer Yudkowsky writes as if he thinks it's 100%, or at least very close to that... https://twitter.com/ESYudkowsky/status/1462960741049200643
It's his usual style on any issue regardless of his level of inner confidence. Whether it's rational, or whether Twitter is a platform particularly suited to rational discourse is of course a matter of opinion.
I have the impression that one major function of the FDA is to shield prescribers from liability. If the FDA says something is safe under X conditions, then you can't be sued if you prescribed it under X conditions and something went wrong. If the FDA says something is efficacious for Y, then you can't be sued if you prescribed it for Y and nothing happened.
Tort is not great. It seems like a good thing to be able to hold doctors accountable for mistakes and biases that cause health damage, but at best Tort adds friction and conservativeness to healthcare, and at worst it aids bad actors and hurts good ones.
But when I try to think about how to fix tort, one of the first ideas I have is tort insurance. But it seems like private tort insurance would be really hard, and public tort insurance...would look very very similar to the FDA.
And it's not clear that multitiered FDA approval would be able to preserve this tort insurance feature.
I think the idea that Scott and others (Marginal Revolution, I think) have put forward of having more gradations of FDA "approval" is a promising one. Is anyone working on finding Congressional sponsors for legislation that would do that?
I think the especially weird thing is that in the historical analogy of AZT for AIDS, after the trial was stopped early, the FDA issued thousands of Compassionate Use Exemptions so that people could use the drug even while it was getting approved.
I explore that story here:
https://willyreads.substack.com/p/a-history-of-the-fda
So why couldn't the FDA use this precedent?
One possible (but still IMO unlikely) reason delay isn't so bad: logistics of getting high risk people tested within 3 days of symptoms and getting them the treatment, are harder than we think, so Merck and/or healthcare providers are working on a way to roll that out easily.
If the trials were done on people within 3 days of showing symptoms, how well does it work on people who take it at longer intervals?
Is it a case that "works great if you get diagnosed within 3 days, isn't better than aspirin if you get diagnosed 8 days"? Questions like this are what interest me about it.
My understanding so far was, that you need to give it early, in the first days after the onset of the symptoms.
Which is obviously an issue, because most people have lighter symptoms for a couple of days up to a week and develop worse symptoms only afterwards. If this holds true, you can't give the medication specifically to those persons in risk of complications or death, because you don't know early enough who those are. Except general risk factors of course. I remember deliberations on whether to give medication like this as a precautionary measure to everybody with a positive test result in future outbreaks in retirement homes this winter (context is Germany). Or to everybody with symptoms who cannot be vaccinated. I don't remember if they were talking about Paxlovid or sth. else, could check that. There shouldn't be that many options I assume. The underlying assumption of course is: even if there'll be an effective drug this winter, there'll not be enough of it to give it to everybody. It would still make a difference.
There might be a weird backwards sort of logic here where, because the population with AIDS was so heavily stigmatized already, people were more willing to break normal medical ethics rules for them. I can imagine a scenario where more stick-in-the-mud types can't be bothered to care, and more forward-thinking folks prevail. Meanwhile with covid, the stick-in-the-mud folks who never gave a shit about AIDS patients suddenly need to grind everything to halt in case one random boomer has a bad interaction.
I think the appropriate term is "sympathetic", not "stigmatized". In the 1980s, homosexuals were a sympathetic population to the sort of people who e.g. run the FDA. Here and now, the people with COVID are stigmatized - they're a bunch damn dirty antivaxxers, in some jurisdictions excluded from normal social and economic life and/or forced to wear a de facto batch of shame. But they're not at all sympathetic to the Blue Tribe elite.
True, not every COVID patient is unvaccinated. But neither was every AIDS patient gay and/or an IV drug user in the 1980s. And when it served the narrative, we'd highlight the people who got AIDS through e.g. blood transfusions then, just as we highlight the breakthrough infections now. But only so long as it served the narrative, then everyone pivoted right back to AIDS as a plague on the sympathetic gay community, or COVID as a disease of the stupid antivaxxers.
Dr. Marty Makary, who has really become my hero, has an Oct 7th op-ed in The Wall Street Journal arguing the FDA should allow expanded access to Molnupiravir
https://www.wsj.com/articles/fda-covid-19-coronavirus-death-hospitalization-molnupiravir-expanded-access-therapies-11633615268
(non paywalled: https://archive.md/QLo2D#selection-4287.65-4287.500)
He notes there is precedent for this:
"the FDA could allow compassionate use of molnupiravir now by activating what is known as an expanded-access protocol, ahead of the drug’s formal evaluation. There is a precedent. Last year, the FDA activated the protocol for convalescent plasma months before it issued a formal EUA. The agency also allowed monoclonal antibodies to be given to President Trump and other officials last year before that therapy was granted a formal EUA."
he also claims:
"Molnupiravir pills are ready to go. The government has prepurchased 1.7 million treatment courses. The medication doesn’t require refrigeration and is easy to ship. These pills are sitting on a shelf as Americans are dying in hospitals."
Obviously Makary's arguments extend to Paxlovid too.
The manufacturing argument can explain why the FDA wants some more time to check if the mass produced drug is as safe as the trial drug. But it doesn't explain why they would then stop the study. Presumably, Pfizer has already produced enough to finish the study. For the people in the study, having a 50% chance of getting life-saving treatment is unambiguously better than a 0% chance. And we get more data.
I could be wrong but I believe stopping the study means the placebo patients are switched to the real drug - that's what stopping it for efficacy means.
If this is true, then great !
I could be wrong here. If there is anyone who has experience with a trial that was stopped for efficacy, then I would love to hear which is true.
That has always been my understanding of what happens when trials are stopped because the substance is too efficacious - the placebos are all given the real thing too, they aren't just released into the wild.
I don't have any experience but I did find this article on the ethics of truncating trials from 2007 which implies that this is not standard practice (or was not, in 2007):
https://www.acpjournals.org/doi/full/10.7326/0003-4819-146-12-200706190-00009
> "Finally, stopping a trial early does not guarantee that current and potential trial participants will receive the apparently beneficial treatment (assuming that one believes they should). Studies of dissemination of new treatments reveal that long delays, such as those between reports of randomized trials and recommendations of experts in review articles and textbooks, are common (14). Continuing a 2-group trial gives participants at least a 50% chance of receiving the experimental treatment, whereas if the trial is stopped early, the probability that participants will receive the treatment due to rapid dissemination is likely to be considerably less than 50%."
I think I've been imagining treatments for a chronic condition, where all participants are enrolled at the beginning, and half get one treatment and half get the other. In that case, they usually give the placebo group the treatment. But I suppose with the covid trials, they are constantly enrolling new patients and randomizing them to one half or the other, and if they stop, then they stop enrolling new patients.
No. When designing a trial, you have to make a guess as to the lowest size effect you expect to see. You then have to design the size (number of patients and duration) of the trial to give you a pre-determined chance of detecting that size effect. The smaller the size of the effect, the larger the trial has to be to reliably detect that effect. If you're lucky and the effect of your drug is substantially greater than the effect you designed for, then your trial will reach your pre-determined significance well before you get to your original trial size. The statisticians monitoring your trial will notice that and say "Hey, you met your significance goal. It's a success. You can stop now", and if the FDA agrees, you stop (because money and there's lots of other things to spend it on to get this drug approved). There's nothing so far about giving treatment to the placebo arm. For some trials, you enroll a bunch of sick people and start giving them drugs, or placebo. If the treatment arm suddenly rises from their wheel chairs and casts off their crutches, the statisticians will give you the nod. But now what do you do with all the sick placebo patients? That's really up to you, but drug companies don't want to be seen as evil, so they will often just give drugs to the placebo arm. I don't think this is an FDA requirement, nor do I think there is some pre-determined criteria for doing that. In the present case, because they wanted to give the drug in a fairly short window after symptoms appear, the patients are enrolled as they present themselves to the trial, not in a big bolus at the beginning. As it appears to be the case in this trial, they got the nod from the stats at some point, but there wasn't some ward with whole bunches of sick people lying around with placebos that you could now dose with the good stuff. Since it's a rolling enrollment, the people who were going to get placebos haven't shown up, yet. They aren't sick yet. So, you just close the doors and say the trial's over. There's no need to keep enrolling people into a finished trial, hence no more people to get placebos, or offer treatment. There may be a few people who arrived in the short window just before the stats called time that might be offered treatment just because they're already in the door. I don't know.
TL;DR Stopping for efficacy happens because you proved it works, not because you have to switch placebo patients to treatment.
But that's not what Scott's original post said. Apparently, trial was stopped early because "it would be unethical to continue denying Paxlovid to the control group". I guess that declaring this doesn't technically mean that the control group would in fact receive treatment, but the alternative seems just a bit too kafkaesque to be true. inb4 being shown otherwise, I guess...
Who said "trial was stopped early because "it would be unethical to continue denying Paxlovid to the control group"???? The press release, which is the only official communication I have see on the subject, says "At the recommendation of an independent Data Monitoring Committee and in consultation with the U.S. Food and Drug Administration (FDA), Pfizer will cease further enrollment into the study due to the overwhelming efficacy demonstrated in these results and plans to submit the data as part of its ongoing rolling submission to the U.S. FDA for Emergency Use Authorization (EUA) as soon as possible." There is >nothing< in that statement about ethics or giving anything to the control group. Scott's original post may have said that, but having read the references in Scott's original post, there's nothing authoritative that says anything like that. Scott does point to a Tweet: https://twitter.com/KelseyTuoc/status/1461781455407828993 that claims "they'd stopped enrollment into their Paxlovid study because it prevented 89% of Covid deaths, making it unethical to not give it to controls.", but there's no reference given to back that up. So, my take is that Scott just blindly repeated some rando's uninformed tweet and everyone who thinks Scott is a careful scholar of drug trials has taken it as gospel. Scott also says "Perhaps there’s not enough evidence for the FDA to be sure Paxlovid works yet? But then why did they agree to stop the trial that was gathering the evidence?" As I've pointed out in other comments, Pfizer has 10 other human trials, 7 ongoing, that are still gathering evidence. So, in fact, Pfizer has not stopped all trials gathering evidence, a fact that Scott, or anyone else, could have found in 5 minutes by looking at clinicaltrials.gov.
That does indeed seem like a significant mistake on Scott's part, and significantly erodes the rhetorical point he was making.
I love internet comments who know what they are talking about.
There is also an unfortunate tendency for people who are experts in one thing to assume that the experts in another thing are idiots. A random example might be a consumer electronics engineer talking s_it about automotive electronics engineers because they still do X. And the automotive engineer might say, "Yes, we could do it your way. If we were designing cars to be used indoors. We need products that work as well when parked outside all night in Alberta in January all they do when parked outside all day in Dubai in August." Ohhhhhhh....
> the automotive engineer might say, "Yes, we could do it your way. If we were designing cars to be used indoors. We need products that work as well when parked outside all night in Alberta in January all they do when parked outside all day in Dubai in August."
Do they? I would estimate the number of people interested in that use case at somewhere between zero and one. It's just not that common to move cars overseas.
The manufacturer might get economies of scale by producing the same cars for Alberta that they do for Dubai, but that's not a business need (there are plenty of businesses that sell more than one product!), and from what I've read, it's also pretty unlikely that a car legal in one country will meet the legal requirements of a second country.
I think you'll find that the vast majority of high-volume cars outside the US market are very much designed to be sold and operate in multiple countries. If I buy a BMW in Australia, it will meet the Euro emissions standards, and it will not be significantly different to the same model sold in Sweden.
Individual consumers may never take it between the two extremes. But every car on the lot could end up in either extreme and consumers don't want to care to pick only certain kinds off the lot.
Just driving from Seattle to Phoenix in spring can take you through extremes of wet, dry, heat, and cold.
Even if we just focus on the continental U.S. market, changing the original statement to "Minneapolis in January" and "Phoenix in August" makes the same point.
I'm not sure if I buy the mass-production-check reasons given.
To be a big drug manufacturer, it seems like you must already have very robust post-production quality control processes in place, and these are going to look more or less the same for any drug. For each batch, pull X number of random samples, check that they look like the batch from the trial and that variances are sufficiently tight, etc. This process, while not foolproof, seems like it should at least easily be strict enough so that some extra perusing from the FDA will have no marginal utility. That is, if the problem is subtle enough to pass the pre-existing post-production QA check, it will be too subtle for the FDA to detect. (And of course the FDA could take random samples in real-time and run it through their own lab.)
Is anyone aware of a historical case where the drug company said "Hey, we're ready to go" and the FDA came along and said "Doh! Vat 76 in production line 4 has too much temperature variance"? I'm skeptical.
With regards to how well the manufactured drug maintains its efficacy in storage, it seems like a provisional approval could just include a requirement that the initial expiration date is no further away from manufacturing than those used in the trial.
A couple thoughts.
1) Having worked in a production environment (though certainly not anything pharm related), I bet something like that happens semi-regularly (either issues are found during the FDA review, or issues are found by the company while preparing for the FDA review which is just as good).
2) If the FDA isn't doing these reviews, the "existing" QA checks probably don't exist, or at least wouldn't be nearly as thorough.
Eh, maybe.
To take my personal anecdote as an example as well, I work at a black box trading firm. Obviously, we have tons of checks at many levels to prevent bad strategies from going to market. Are they foolproof? Certainly not. But I'm pretty confident that there's no way that some outside group of even really smart people could come in and find a bug without basically becoming a full-time employee, and even then it would be very hard because, obviously, the code was written by tons of smart people who are extremely worried about a bad strategy hitting the market.
I do think an outside group could analyze our outputs and find signs of a problem, if not the exact cause. It wouldn't be easy, but you could look for clearly infelicitous patterns and whatnot. (Obviously, we do a lot of this ourselves, too.)
Similarly, I think any value-added from the FDA on QA would have to come from analyzing the outputs of the manufacturing process, i.e., the actual drugs, which I think could be done concurrently with approval.
I'm not sure what you mean by bug.
As an example, back in 2015 the Swiss Frank made a 58 sigma (?) move against the Euro and various people lost their shirts. The issue was the math said a move of X should only occur once in 200 billion years or whatever. But obviously that had nothing to do with the Swiss National Bank's decision to intervene in the currency markets.
I can totally see an organisation falling into group think because some core part of their model has worked so well in the past.
Oh, sure. But no outside group would have ever gone in and been able to assess that any better. The outside group would be just as subject to group think and had lower incentive to quibble over the math.
I don't think the idea is that regulators would look for bugs directly. Rather, the regulator makes sure *you* are looking for bugs, by requiring evidence in the form of documented processes and excessively long reports.
I'm not sure if hedge funds have the same requirements, but AFAIUI this is basically how it works at banks.
In your case, the incentives might line up well that you do lots of QA. But that's not always the case, and that's where there's opportunity for value-add from regulators.
The results on Boeing's QA after they were allowed to do the regulatory parts in-house suggests that having a regulatory set of checks, while not necessarily directly useful, may incentive-wise be a necessary evil nonetheless.
> Is anyone aware of a historical case where the drug company said "Hey, we're ready to go"
> and the FDA came along and said "Doh! Vat 76 in production line 4 has too much
> temperature variance"?
There are FDA warning letters that say: "On inspection, we found pigeons were roosting in the cleanroom in your facility" or the like, but those aren't relevant here. Instead they will ask "How hot can Vat 76 get before the batch is ruined, and you have to tip it down the sink?" Possible answers are, in order of increasing acceptability:
-"I dunno, we always do it at that exact temperature, and nothing bad ever happens"
-"I dont know the exact temperature where it goes bad, but this one time when it was 50 degrees over, the batch went crispy on the bottom like a paella"
-"We tried it at a range of temperatures, and at 10 degrees too hot you start to start to see a new impurity forming"
-"We modeled the interaction of temperature, concentration, and pH, and we can show you that any combination of these that lie within this 3D Design Space will give you an acceptable product"
If all your answers are like the last one, your manufacturing process will be approved. Anyone can notice if something went outside spec during manufacture; the FDA want you to justify why you chose that spec in the first place, so you can prove that your choice of temperature control methods was correct.
How often does this matter for new drug approvals? The best answer I have is this paper:
"Scientific and Regulatory Reasons for Delay and Denial of FDA Approval of Initial Applications for New Drugs, 2000-2012"
https://dx.doi.org/10.1001/jama.2013.282542
302 new drug applications were examined, of which half were approved immediately, and another quarter of the applications had "deficiencies" to be resubmitted before ultimate approval.
"We limited our analysis of [manufacturing] and labeling deficiencies to
products that were not approved despite satisfactory safety and
efficacy (n = 23). Chemistry, manufacturing, and controls deficiencies
included problems with dissolution or manufacturing
specifications, incomplete stability data, high endotoxin
levels, and deficiencies noted during inspection of manufacturing
facilities."
Note "endotoxin" is a component of bacterial cell walls. Its presence in your drug, besides leading to unwanted immune reactions in patients, indicates that at some stage you left the batch sitting around long enough that bugs started growing in it.
Results: 17 drugs of 23 failed the first cycle review due to chemistry, manufacturing and controls issues, two for CMC plus labelling; the other 4 were arguing about the labelling alone. However, most of these were eventually approved (i.e. the problems were fixable). Only 4 were never approved, and I would bet some or all of these were just because the pharma company ran out of money/interest and stopped trying.
Regarding stability, we saw the impact of this with the emergency approval of the Pfizer vaccine. Remember how at first it needed to be stored at -80 celsius, leading to supply chain issues all over? But then after a while Pfizer announced it was ok at higher temperatures (I forget the details) and you could leave a vial thawed at room temp for a little longer before use. That shows that Pfizer didnt have all the stability data ready, but the FDA let them go ahead anyway, as long as they were super conservative with their choice of storage temperature/shelf life.
I'll also comment on this:
> To be a big drug manufacturer, it seems like you must already have very robust post-
> production quality control processes in place, and these are going to look more or less
> the same for any drug.
I'll blame TV for this one, where the hot young chemist squirts a sample of drug into a machine, then walks round to the printer and rips off a printout of everything thats in the drug sample.
In reality, with a new product that noone has ever made before, the chemical properties of the product itself and the impurities and degradants is unknown. With a known product you can buy a certified standard that says purity >99% on the label, and a list of whats in the other 1%, and compare your own material to that. If the compound is brand new, you have to invent the purity test yourself, show that it works reproducibly enough that you can certify the results are true, and justify that process to the FDA. Then guess what the possible impurities are, make or buy samples of all those to compare, and invent robust tests for those too. Then figure out the chemistry of how the impurities got there, how to purge them, and how to prove they are gone from the final product. The whole time you could have got it wrong, and there was something hiding in your original "pure" sample that you never found. I'll illustrate this with more on the ranitidine story I mentioned earlier (apologies if I get any details wrong, I've never worked on this myself, only read about it in the news).
Ranitidine (Zantac) has been used since 1981, and was the world's biggest-selling prescription drug by 1987. It was considered extremely safe and people took it daily for decades. Countless manufacturers must have made batches of this stuff over the years, but noone discovered until 2019 that if you keep the stuff lying around, it possibly decomposes into something carcinogenic. This is the chemistry of the drug itself, not due to manufacturing problems as far as is known, but its still an unsolved problem. See FDA website: https://www.fda.gov/drugs/drug-safety-and-availability/questions-and-answers-ndma-impurities-ranitidine-commonly-known-zantac
"Q. How long has NDMA been in ranitidine?
A. FDA does not have scientific evidence to determine how long NDMA has been present in ranitidine products."
How did this happen, with all the quality control and testing of every batch ever made? The answer is, there was no test for NMDA so noone tested for it, the levels are small enough that it was hidden, and all the batches were certified "pure" and safe. It was only discovered by chance, through a different drug called valsartan. Clever manufacturers figured out a new way to make valsartan that was much cheaper than the old process, thus giving an edge in the highly competitive generics industry. The new process used sodium nitrite, which is well known to form nasty carcinogens e.g. by reacting with amines naturally found in meats etc. The valsartan manufacturers knew this, but since their process had no amines in it, they knew it was safe, and were easily able to justify this new process to the FDA. The problem came on scale-up, when recycled solvents were used, that came from a process where amines *were* used. The solvents were purified, but traces of amines remained, and they didnt test for them or control them adequately. This meant batches of valsartan were released that had relatively huge quantities of NDMA in them, enough that they behaved weirdly, and so someone investigated after the fact. (Insert quote here from Asimov: The most exciting phrase to hear in science, the one that heralds new discoveries, is not “Eureka” but “That's funny...”). The valsartan batches were quickly recalled, a new test for NDMA was invented, and valsartan users were safe again. But of course, now that testing for NDMA was routine, people wondered what other drugs might be contaminated, and the rest is history. The problem with ranitidine is that the required amine is present in the molecule itself, along with the nitrite, and the FDA is worried enough about conditions where the two might meet, that the product has been recalled until this is sorted out.
> In reality, with a new product that noone has ever made before, the chemical properties of the product itself and the impurities and degradants is unknown.
My first thought was that mass spectrometry might be an "easy mode" first pass check for impurities and degredants, as it would highlight anything in the sample that doesn't have the same molecular mass as the things you expect, so you could follow up with chemical tests to see what those things are with the candidate space significantly narrowed by their known mass.
The I realized there were at least a couple huge problems with this. The first being that starches are very common intended inactive ingredients in pills, and starch chains can be any multiple of the constituent sugar's molecular mass.
The bigger problem is going to be isomers and related molecules of near-identical mass. These seem like they'd be very common impurities in manufacturing an organic molecule, and they'd be hard or impossible to differentiate from the intended product on a mass spectrometer.
The common analytic techniques are:
- HPLC-MS: this is a chromatography column hooked up to a mass-spec machine. If you're flush, you do a fragmentation mass spec, where you get both the mass of the original molecule and then hit it with electrons to break it into pieces and get the masses of those as well. This works very well if you know what you're looking for.
- NMR: nuclear magnetic resonance. This is how you figure out what is present if you don't know what it is. If you have a common contaminant, you'll purify it with some kind of chromatography, then run an NMR on it. The magnetic resonances of the hydrogens (or occasionally carbons) give you very detailed chemical information.
So if you know what the desired product is, HPLC-MS is an easy way to measure how much of what you want is there. If you have contaminants, NMR is the common way to figure out what byproducts you're getting, and then you can figure out how much of a problem that is.
Related molecules of identical mass are not much of a problem with the chromatography. Even isomers will get caught out with fragmentation mass spec pretty easily. You can occasionally have problems with racemic mixtures (e.g., molecules that differ from each other at only a single chiral center). Racemic mixing is a problem, for example, in methamphetamine manufacturing.
Even there you can do, e.g., circular dichroism spectroscopy to evaluate it. (Look at how polarized light rotates through the solution, which depends on the chiral centers).
This seems like an unusually fast turnaround time for highlights-from-the-comments. Or maybe it just seems that way because we got two such posts in short succession and I don't remember how long ago the Ivermectin post was.
The comments were so good that it was unethical not to post them early.
LOL
There is indeed real urgency to this situation. Unfortunately I don't think any of us have a good theory of change as to how to actually change things, but screaming into the wind is still better than nothing.
This whole thread seems to be predicated on the misconception, from Scott on down, that this efficacy trial ( NCT04960202) is THE trial that will determine if the FDA can authorize / approve it. That is simply not true. There are 10 more, 7 ongoing, human trials registered at clinicaltrials.gov, each designed to test a different aspect of efficacy and (mostly) safety. NCT04960202 was the big efficacy trial, and passing that is a huge milestone in the approval process. (Sometimes you can rescue a efficacious drug that has some bad safety signals if you have a clear understanding of the harmful mechanisms, but I've never heard of of a non-efficacious drug that was brought back to life.) But, safety is still a big hurdle in the approval process. In addition to the registered human trials, there are dozens, if not hundreds, of animal studies, that you and I will never see, that also go into the FDA decision process. Nobody here has any idea what the results of those are. (OK, we know that not all the rats died, or the human trials would not have taken place, but there's a lot of room between there and "its as harmless as distilled water, or rainwater, and only pure-grain alcohol".) The results of all those studies runs to hundreds of thousands of pages, and anyone who claims FDA can go through all that in 24-48 hours without cutting corners is talking nonsense. Basically, the press release that comes after a good efficacy trial doesn't tell you much about the real state of the approval process, or when you should start grousing about the FDA dragging it's heels.
The fact that the efficacy trial was halted merely indicates that FDA is satisfied that you checked that box in the approval process and that Pfizer can put its resources to checking the other 99 boxes it takes to manufacture, market, and distribute a new drug to patients. To halt a trial for futility means FDA will never approve this drug no matter how well the rest of the trial goes and you should just stop putting people at risk.
This is a good thought, but if this is correct, I would no longer think that BronxZooCobra's above reply to TheChaostician has force. If the FDA is still reasonably worried about safety, that I think it's hard for them to reasonably say "we already know there is such a huge, positive expected value that it would be unethical to give a placebo any longer so we have to stop the study".
FDA didn't force anything. "At the recommendation of an independent Data Monitoring Committee and in consultation with the U.S. Food and Drug Administration (FDA), Pfizer will cease further enrollment into the study due to the overwhelming efficacy..." Basically, FDA concurred with Pfizer that they showed efficacy and they don't need to do any more to convince FDA that it works. That actually can speed up approval since nobody's waiting around for the results when the outcome is already known. I don't think halting the trial has anything to do with the patient's well-being. But, there are still more safety trials to complete, so it's not fair to say that "we already know there is such a huge, positive expected value" because the other side of the "expected value" calculation, i.e. safety hasn't been determined, yet (Maybe. I don't really know what that state of approval really is, but nobody else here knows, either.)
Yeah, but regardless of who forced what, there's still an inherent contradiction, IMO.
If what BronxZooCobra said above is correct, stopping the trial means they start giving the placebo arm participants the real drug. Well, if it's OK to give the placebo arm participants the drug, why isn't it OK to give everyone else the drug? Obviously, trial participants may systematically differ from the general population in terms of comorbidities, other medication, etc., but if so, it seems you should just go with Scott's idea of a phased approval where you only give it to members of the general population who satisfy the trial inclusion criteria.
"The results of all those studies runs to hundreds of thousands of pages, and anyone who claims FDA can go through all that in 24-48 hours without cutting corners is talking nonsense"
This just raises more questions:
1. How many of those pages contain information that's actually important?
2. How often does the FDA spot problems in one of those hundreds of thousands of pages? How often are those problems serious?
3. How many people did the FDA commit to going through those documents? Why not more?
4. Is the FDA working 24/7 on this approval given the number of people that could be saved or are they taking time off? If they are taking time off rather than working in shifts, why is this the case?
5. Where's the cost-benefit analysis on all of this given that this drug is primarily designed for high-risk unvaccinated patients who have a 10-50% chance of getting hospitalized otherwise and a 5-30% chance of dying?
6. How much of the work could've been done by the FDA in advance rather than waiting for the trial to be over?
7. How much of the work is repetitive and could be parallelized or skipped altogether for someone as bigger as Pfizer?
8. What's the SLA at each layer of interactions between Pfizer and the FDA and how much do communication latencies affect the time required to grant approval?
9. How many pills could Pfizer churn out if the FDA approved it right now? Are we actually waiting for FDA or are they still faster than the manufacturing process?
10. Why isn't approval of drugs separated into approval of an "ideal substance" and approval of the manufacturing site?
My experience with regulatory processes is limited, but nonzero. My general observation is that every part of the process has some justification (e.g. something that has actually gone wrong before), but putting it all together, it becomes ridiculously overbearing. Regulatory processes are trying to guard against human error, but in doing so, lose all the efficiencies of human judgment.
I'm not sure I see a way around it, though. When something bad happens, everyone demands "there should be a process to ensure that can't happen again!", it seems reasonable, and so yet another item is added to the checklist.
To give a non-answer for #1, if you're aiming for multiple 9's of safety, you are necessarily going to have to check an awful lot of unimportant details. I both expect and would hope that the reports have lots of unimportant details discussed at length.
To give a non-answer for #2, it's not just the errors you spot, it's the errors you avoided in the first place. Having a strict process means corners won't be cut in the first place, because they know they can't get away with it.
I seem to remember a post back in the SSC days where Scott complained that the FDA is loathe to rescind approval of a drug once it is given. That even fast-tracked drugs, approved provisionally based on surrogate endpoints, don't get pulled from the market after not doing the appropriate follow-up studies based on hard endpoints.
What's the probability that an emergency authorization of this drug will be approved, save some lives in the short-term, and then persist in medical practice for years before its use finally fizzles out? Will we save more lives on the near side of the distribution than we risk in the perpetual long tail?
And remember that trials stopped early on ethical concerns for the placebo arm consistently overstate efficacy - which is exactly what we would expect from the statistics. Not saying the efficacy isn't there, just that we should presume it's overstated based on the declared early end of the trial. Which takes a little from the sails of the argument for rushing it through a few weeks early.
Scott complains about the system we have, with some merit, but approval in the current system is what it is. A better system would allow tiered approvals like Scott suggests, but we're talking about approval the current system. Is there evidence of a strong - functioning - corrective mechanism post-approval? Or if this is a mistake, do we just have to 'live with it' from now on?
> Also, I am a bit surprised that Scott seems to use "expected value" so freely in this context. Medical ethics does not seem to work that way at all? At least, there seems to be a heavy asymmetry between action and non-action, maybe that is reflected in the approval system as well?)
That's precisely the problem.
> I agree something like this is true, which is why my preferred solution is for the FDA to have different levels of approval.
Isn't this already true? There's one level of approval for "medicine" and a different level of approval for "supplements".
SSC has discussed the different levels in the past: https://slatestarcodex.com/2013/09/28/sleep-now-by-prescription/ https://slatestarcodex.com/2014/06/15/fish-now-by-prescription/
Where are the ritonavir supplements?
Thanks for the post, and the highlight :) Regarding your question:
> We know that Pfizer has already started making the drug; is it possible they can run the factories
> now, the FDA can examine the factories while they’re running, and then the FDA can retroactively
> pronounce that the factories are fine and people can use the drug they produced?
In short, yes. The FDA calls this "Concurrent Validation". Quoting from here: https://www.fda.gov/drugs/guidances-drugs/questions-and-answers-current-good-manufacturing-practices-production-and-process-controls#5
"... after having identified and establishing control of all critical sources of variability, conformance batches are prepared to demonstrate that under normal conditions and operating parameters, the process results in the production of an acceptable product. Successful completion of the initial conformance batches would normally be expected before commercial distribution begins, but some possible exceptions are described in the CPG. For example, although the CPG does not specifically mention concurrent validation for an API in short supply, the Agency would consider the use of concurrent validation when it is necessary to address a true short-supply situation ..."
An Emergency Use Authorisation would be another exception. If you can put your hand on your heart and say "I believe these batches will be ok, based on this data I have" then the FDA will listen. They will review your data for themselves though, before giving you the go-ahead, and that is probably whats going on right now for Paxlovid.
"
Dr. Marty Mackary [sic] says this : "As a Johns Hopkins scientist who has conducted more than 100 clinical studies and reviewed thousands more from the scientific community at large, I can assure you that the agency’s review can be done within 24 to 48 hours without cutting any corners."
"
I remember reading this when it came out, and I'm not sure it's credible. I suspect the 24-48 hours is an estimate for an academic review, not a regulatory review. I don't have any knowledge of the FDA's internal processes, but normally a regulator can't just say "looks good to me", they have to file hundreds of pages of reports demonstrating that every item was scrutinized and considered.
Do they have to file those reports before they issue the decision?
Everyone involved *should* be able to write and/or review a draft of all those reports well in advance, with everything but the final numbers and e.g. the case reports of the people who develop significant adverse reactions in the last week of the trial. And it should be possible to make an approval decision contingent on the final numbers coming within a specified limit. We do that routinely in e.g. space launches.
Filling in the final blanks and making sure they are in range, should be a 24-48 hour task if you need it to be. But it wouldn't surprise me if parts of the bureaucracy would say "That's not how we do things around here, so NO".
In the short term, the pills may do more harm than good: if we start with a limited supply, it would make sense to prioritize the pills for the unvaccinated, since they are much more likely to have severe outcomes. If this becomes a part of the pill allocation process, it would look super awkward, as if we are punishing the vaccinated for doing the right thing. If some people are on the fence about getting the vaccine, knowing that getting vaccinated might hurt their chances of getting the pill might push people towards not getting vaccinated, and if this happens to a significant degree it could outweigh the benefits of a limited supply of the pills.
Read a piece published today in Stat about challenges and obstacles anticipated in deploying Paxlovid and molnupiravir, including remaining questions about how diagnosis may impact when and whether they can be prescribed: https://www.statnews.com/2021/11/23/covid-antivirals-pandemic-game-changers-americans-struggle-access/. It sounded like answering these questions from a regulatory standpoint might be a major contributor to the time needed for the FDA to approve Paxlovid - because it sounds like they need to issue guidance on whether a PCR positive Covid test is needed, or for which populations it may not be, or if it can be prescribed later than three days post-symptom onset (because the study only covered how well it works when given within three days of symptom onset). One person quoted suggested that the drug(s) be authorized for prescription some immunocompromised people preemptively in response to Covid exposure without a positive diagnostic test. But with only the data available from the studies at the time of the FDA meeting, it sounds like the FDA needs to be able to issue guidance on all these things, many of which which are not directly answered by study data alone (and ideally make the guidance mostly feasible for real-world conditions where e.g. Covid testing turnaround time may not be very fast). Am I correct in assuming this is a major contributor to the time to approval in this process? These seem to pose a real challenge (given real-world limitations in distribution of and access to the drug, as well as safety considerations - even if you had unlimited supply, the cost/benefit ratio for preventative prescription of Paxlovid for Covid exposure or suspected Covid might not pan out in all populations just because post-diagnosis treatment does) that consumes time for reasons other than bureaucracy just takes time because it has to take time even in emergencies.
This is exactly what makes me so mad about arguing on these issues. The same people who defend the legal regime that totally bans drugs which the FDA hasn't yet found to be sufficiently beneficial (tho often claim to believe there is a constitutional right to autonomy which lets women choose to abort babies, which I love the idea of but surely that creates some similar right to take unapproved potentially lifesaving meds) then use the worries about lack of trust to justify being extremely reluctant to approve anything.
It's a vicious bootstrapping which seems largely based in the inability to imagine that people could adjust to any other message. Sure, now we may equate FDA approval with certain features but there is no reason the FDA couldn't wake up tomorrow and tell ppl only green boxed meds are tested to that degree and (with a bit of PR) most ppl and doctors would adapt.
>Scaling up chemical manufacturing is not trivial (a regular contender for Understatement of the Year). E.g. heating and stirring work differently in different sized reactors. Heat transfer in and out of your reactor works through surface area, but heat produced/consumed by the reaction depends on volume. If your stirrer design isn't right for the viscosity of the solution, you might get hotspots and so on.
>Ideally, the FDA expects you to understand the chemistry so thoroughly that you know everything that can possibly go wrong, and design your commercial process so that none of these things can possibly happen. The commercial batches will therefore be identical *by design* to the clinical trial batches, and you have to prove this with science. Of course in practice you don't have to have collected all the evidence before you can start selling batches, but you must have your plan in place with a solid scientific justification for every decision you made along the way. It also helps to have made a statistically valid number of commercial batches to show that your beautiful process works as designed (how many batches is that? you tell me, and justify your decision).
>Pfizer/Merck will have thrown everything at this problem alongside the clinical trials, as they can afford to do this, so their regulatory submissions will be pretty good. However they still might have to store the new batches for a few months to demonstrate that they have a comparable shelf-life to the old batches, and FDA might wait to see this data etc.
I do not understand why the FDA would be concern itself with the details of the manufacturing process when they can just require a chemical analysis of a representative sample of the product.
I get that for mRNA vaccines, this may be insufficient because you also have to check the size of the lipid nanoparticles and that the mRNA is actually within them, but for the Paxlovid, Wikipedia suggests that the active ingredient PF-07321332 is just a straightforward organic molecule.
I am not a chemical engineer, but between mass spectroscopy, chemical analysis (for isomers) and in vitro testing (?), I feel that modern age chemistry might be up to the task to decide how identical two pills are two each other.
I feel the selection of a process of manufacturing should just be a business decision: giving a probability estimate of a certain process chain to fail and ruin your bunch, optimize for maximum expected gain, or something.
The same goes for the shelf life. If the pills are sufficiently identical to the trial ones, I would assume that their shelf life is similar as well. All the counterexamples I can think of feel rather constructed, like "We replaced Phosphorus 31 with P-33, now our chemical has a half-life of 25 days" (and skipped mass spectroscopy, and also the chemical costs three orders of magnitude more) or "we put H2O2 into the filler material, and given time, it reacts with the active ingredient over time" (so the pills were obviously not chemically identical).
If the product is easily chemically analyzed, it should be enough to regulate the product: When I buy table salt, I do not really care if it comes from sea salt or the salt mines or from someone burning sodium in a chlorine atmosphere.
Only if the product is a complex mixture of different chemicals regulating the process chain makes sense: a complete microbiological analysis of a representative sample of soy burgers out of ever freezer is not cost effective, so we mostly regulate the manufacturing process, cold chain, and use-by dates and sample only sporadically.
I feel standard chemical drugs (e.g. not mRNA vaccines or the like) are mostly the NaCl case, not the soy burger case.
That is not to say that upscaling the synthesis of new chemicals is easy, only that trial and error should be a valid strategy. Any new drug even moderately effective against COVID-19 stands to make billions. If the manufacturer has to throw away the first few millions of doses because the selected process did not work, this should not affect their bottom line too much beyond the added delay to market.
> I do not understand why the FDA would be concern itself with the details of the manufacturing process when they can just require a chemical analysis of a representative sample of the product.
Because then you don't know the failure modes of the manufacturing (e.g., lower efficacy might be acceptable, a poisonous byproduct would not be) and how likely they are. The FDA can't test every batch of product.
> I do not understand why the FDA would be concern itself with the details of the
> manufacturing process when they can just require a chemical analysis of a representative
> sample of the product.
Its firstly because no chemical analysis measures absolute purity, you only get a measure of something else plus an associated experimental error. Each separate test or comparison you do adds further assumptions, until the final combined results are just an approximation of the "true" purity/potency. In mass spec for example, every component ionises differently, so your quantitative work has to correct for that. Some components don't ionise well, or not at all, and if its something new that you weren't expecting it could be hidden in the noise. Other components are too light or easily fragmented or swamped by matrix or whatever. Then there's always that one test that just doesn't work very well because the compound is hard to analyse, and so its best to leave that for Alice to do on Tuesdays when shes in, since she is the only one who can get the instrument working optimally. The whole thing is subject to the same biases and flawed conclusions as anything else in science, which of course the FDA requires you guard against (e.g. the people who measure the numbers must be employed in a separate department of the company from the people who check the numbers), but its extremely difficult to argue with a determined fraudster who is only required to provide evidence that they did the required testing. Its much harder to fool yourself (in the Feynman sense) or game the system if you have to provide evidence that you understand how to make the product correctly to begin with.
Secondly, if you have a problem with quality, no amount of increased testing of the final product is going to fix that. You can't test quality into a product; you have to design it in. You're correct that trial and error is a valid way to do this, but its better for everyone if you do this systematically from the start, rather than waiting for a batch to fail. From the FDAs point of view, its a waste of resources for them to review your drug a second time, after you didn't do it properly the first time. Its dangerous if you have to recall a batch that passed all the testing, but then you figured out there was actually something wrong with it after you already sold it (see valsartan story above). Its wasteful and perhaps unethical to have to repeat some of your clinical trials, because you figured out that your commercial process was fatally flawed and you could never get your new batches quite the same as the clinical batches.
"Maybe at stake is not the reputation of the FDA, but the reputation of the approval process?"
This but cynically. Every time the FDA drags its feet part of me thinks that they're protecting the reputation of their insane bureaucratic obstructionism. If they ever let a high-profile pill through with less than six weeks of meetings and nothing terrible happened, people might start to wonder why we're paying so many bureaucrats to sit in so many meetings.
The only reason I'm not convinced this is what's going on is that it implies people pay enough attention to the FDA to notice what they're doing.
Seems plausible, except swap unspecified people for other rival bureaucrats. Each agency surely strives to present itself as doing indispensable highhly demanding work, so when the budgetary pie gets divided they would receive an appropriately large share.
Since you took a lot of trouble to come to a numerical conclusion about Ivermectin, I'm curious how certain are you that the FDA's Paxlovid approval process will cost tens of thousands of lives. Eliezer Yudkowsky writes as if he thinks it's 100%, or at least very close to that... https://twitter.com/ESYudkowsky/status/1462960741049200643
It's his usual style on any issue regardless of his level of inner confidence. Whether it's rational, or whether Twitter is a platform particularly suited to rational discourse is of course a matter of opinion.
I have the impression that one major function of the FDA is to shield prescribers from liability. If the FDA says something is safe under X conditions, then you can't be sued if you prescribed it under X conditions and something went wrong. If the FDA says something is efficacious for Y, then you can't be sued if you prescribed it for Y and nothing happened.
Tort is not great. It seems like a good thing to be able to hold doctors accountable for mistakes and biases that cause health damage, but at best Tort adds friction and conservativeness to healthcare, and at worst it aids bad actors and hurts good ones.
But when I try to think about how to fix tort, one of the first ideas I have is tort insurance. But it seems like private tort insurance would be really hard, and public tort insurance...would look very very similar to the FDA.
And it's not clear that multitiered FDA approval would be able to preserve this tort insurance feature.
I think the idea that Scott and others (Marginal Revolution, I think) have put forward of having more gradations of FDA "approval" is a promising one. Is anyone working on finding Congressional sponsors for legislation that would do that?