Thank God no. The solution to a clumsy but powerful bureaucracy is not to create an entirely new level of even more powerful bureaucracy and hope that, against all experience, it will be sleek, intelligent, and efficient.
There's a child's rhyme about this, about an old lady who swallowed a spider to catch a fly.
Whilst I agree in principle here, what is being advocated in effect is having a government-funded scheme with no mechanism for controlling costs. Or to put it another way, a way for the cartel of approved manufacturers to take money from tax payers. You need to have a check on that process even if it is additional bureaucracy as the costs are going to be less. Ideally though I guess the check should be democratic rather than bureaucratic.
Yes. The only effective restraint on government is active and informed voters. Asking government to keep an eye on government is deeply silly, and never works. I will say a kind word for federalism, though -- having roughly equipotent goverments keeping an eye on each other has some real value, or used to, back when federalism was taken more seriously. It also gives citizens the ability to vote with their feet.
I'm British. I think we're largely happy with NICE, both the population in general and the doctors. They are slow to react, but the general principle of 'government only pays for cost-effective treatment and cost-effectiveness means price less than "£X/QALY" seems pretty sound.
If you guys had the FDA to say "safe enough to be used", and a NICE equivalent to say "useful enough to pay for centrally", your above problem would be solved.
Glad you like your system. I wouldn't have it if you paid me. The only person I want to decide whether a given drug is useful to me or not is me, and I'm quite happy to pay for it, or buy insurance in advance to cover its cost, in order to have that right. I'm gobsmacked the Brits, historically pretty stout defenders of their personal liberty, are willing to turn over that much authority to Parlaiment. Why not have them arrange your marriages while you're at it, eh?
What system does your insurance company use to decide something is "worth it"? Do they do all their own $/QALY calculations each day? Can you compare one insurance company's "worth it" against another's "worth it" to decide which is more in line with your values?
NICE only determines what standard NHS care will cover.
"I'm quite happy to pay for it, or buy insurance in advance to cover its cost, in order to have that right" would absolutely still be an option.
People often seem to act like the existence of the NHS somehow means that private care and private health insurance don't exist over here, and it simply isn't true (I think Canada is closer to the strawman I keep seeing people attack?) - a more accurate model would be to consider the NHS as Universal Basic Insurance.
Is that the primary goal, though? To do "Science"? Seems to me like it has a relatively large "cost-benefit analysis" role as well, which, as the voting citizens of the country that created the FDA, and who will bear both the costs and benefits of those analyses, I'm not sure it's the "man on the street" unjustly intruding on the FDA's "scientific" jurisdiction. It seems more like the FDA unjustly (and without the relevant expertise) intruding on citizens' implicit demand that their public health agencies serve their best (again, implicit in "best" is some kind of cost/benefit analysis) interests.
Not unless it includes some experiment, and I'm hard-pressed to see what that experiment would be. As an actual real-life scientist, I'm routinely annoyed to see almost anything that involves numerical calculation and/or logic called "science" on the grounds that this is what "science" is. That's mistaking the tools for the task. It's as absurd as saying algebra can be classified as a form of poetry, like Shakespeare, because it (algebra) involves the meaningful arrangement of letters of the English alphabet.
Nothing is a science unless it begins with discovery through experiment. Anything else might reasonably be called philosophy, even natural philosophy, but if there isn't an experiment at the base, it's not science. And for that matter, the use of mathematics is not required: if I hypothesize that it rains every other Tuesday, and conduct the experiment of going outside and returning to drip, or not, on the calendar, I am doing science and my results -- the calendar with rain spots on certain days -- are scientific results, despite the utter absence of math.
I think it’s complicated though. The FDA regulates the trials necessary for scientific investigation of drugs, and in that sense very much participates in the process of drug discovery. They also require complex opinions and decision making from subject matter experts in the scientific fields. So they’re not really “doing science” in the creative sense, but they employ scientists to direct and regulate the process of science (of evaluating and using drugs and other medical interventions). Science in genera is a slippery word and idea anyway
OP also said they lack “the relevant expertise” and I don’t think that’s true - more that they’re misusing it
Sure, they employ scientists, because they need to understand the science, but understanding the science isn't doing science, any more than appreciating poetry is the same as writing the stuff.
I don't think "science" is inherently a slippery word, but a great number of people and interests have made it so because they want to borrow some of the gloss that empirical science in narrow fields has built up for itself over the last few centuries. It's a common form of linguistic corruption.
Perhaps in some general sense, but in the context of this comment about the FDA, it is not the "science" that they are doing - and it is being explicitly contrasted against the "man on the street's" non-scientific "cost/benefit analysis" approach. "Science" defined so broadly that it includes the explicitly political (in a good way), cost/benefit analysis of "gee, maybe we should accelerate approval of a vaccine, given that thousands are dying each day" is meaningless - and worse, if it is "science," then it means that the FDA is not a scientific organization because it's obvious they aren't doing this - by their own admission.
The FDA is not even remotely a scientific organization. It's a regulatory standards-setting body, somewhat akin to NIST or the standards-setting part of the NHTSA or FAA (i.e. without the investigations arm). They are supposed to assess the evidence given to them about new drug efficacy and side-effects, and decide whether to approve or not the proposed use of the drug. Not only do they not do original science, they are not even like a scientific management committee, because they don't propose new areas of research, they don't favor or disfavor this or that line of research, or this or that research spending priority -- they just look at the applications laid before them and make a judgment on it.
I guess you can call them "scientific" in the sense that theyr'e supposed to (1) understand the science they're called upon to judge, so they need to have a background understanding of science, and (2) they are supposed to make their decisions based on a rational assessment of the measured effects -- and not on political, economic, or philosophical grounds.
This is actually not true. The FDA does have a few smallish groups that engage in actual scientific research, often in collaboration with biopharmaceuticals companies. These groups are kept separate from the regulatory function, but I imagine they likely still interact with the regulators. The FDA also hosts seminars delivered by biopharma experts on emerging technologies. Generally, I've found the FDA folks to be quite scientifically knowledgeable, at least within their own specialty. The FDA inspectors are particularly good; they are both feared and respected in the sense that they are quite good at finding legitimate deficiencies.
My point here, the problems at the FDA don't stem from incompetence or poor scientific knowledge. They stem from bad incentives and a flawed system.
Source: I am a co-author on a paper generated from an FDA/biopharma collaboration, and have been involved with plenty of FDA audits.
I think perhaps what you want to say is "that's true in a general way, but there are interesting exceptions -- they have some small groups that actually participate in research. Plus they sponsor seminars and such." That's a very valuable addition, and I thank you for adding it. But I still think a strong general statement needs to be made, because I suspect there are plenty of regular folks who actually think the FDA directs research, and the fact that it doesn't is important.
Thanks for linking to my piece. An interesting problem you bring up in the Axsome story is effectively a public goods provisioning problem: how do we incentivize good trials for generic drugs for repurposing?
Good trials for generic drugs provide the public good of accurate information on a clinical question, and then anyone can use that information.
Some ideas to make this better, ignoring novel drugs to simplify:
1. making clinical trials cheaper per patient. I don't know enough about this to have specific proposals, but it is something I'm eventually going to learn more about. Tentatively: something involving substantial reductions in the power of IRB's and using tech to make multi-site trials easier.
2. prizes for researchers who successfully discover a new use for a generic drug and prove it with a trial. This exists informally, in that a highly cited research trial paper will give you career capital and speaking invites.
3. Something involved prediction markets and clinical trial outcomes? Maybe people who are good predictors of successful drug repurposing should get $$ to choose more drug repurposing candidates?
I've always liked the idea (using the tiered/unbundled FDA Scott describes) + "human challenge trials", that after we do normal trials to establish initial safety (i.e. you don't die when you take it) then we open it up to a "soft approval" where you can legally take it, but you have to sign a really big waiver saying "I KNOW THIS ISN'T FDA APPROVED AND I WON'T SUE" and also "I PROMISE TO BE A PARTICIPANT IN THE ONGOING TRIALS FOR EFFICACY" and then also pay people enough (maybe make Prestigious Advancer of Science Through Clinical Trials a viable career option) to participate.
For 3, part of the problem is that one can’t really make as much money off of generics? Maybe a smaller but significant monetary reward for the company/individuals that pushed through the trial (like 2), maybe per drug prescription, as that more directly replicated the incentive for new drugs and provides the money right to the people who develop them.
I’m not sure the scale works there, you can’t really generate the same amount of money that a very expensive new drug candidate that isn’t genetic would sell, so it’s still very lopsided. I think 1) is the best option, but could mean many different things - it’d be really interesting if a group of doctors could run a multi center trial “themselves” without needing significant funding or backing.
A public body to ... study repurposed generics? It seems like the sort of thing where the problem space (all diseases) times the solution space (all existing drugs) times the space of domain experts (for specific techniques and diseases and drugs and whatever) times people who have the disease and are willing to take it times the people who can administer and run the trial ... is very large, and would be better with some mechanism where more people and companies can participate if they think they have something good. So you’d want big biotechs and small biotechs and individual doctors or researchers and universities and whatever to all be able to do (I don’t know how you both make it compensated and doable at the high standard of a clinical trial but it’s worth thinking about) the sort of repurposing research here. Or maybe federal funding for research in it like public research, paying people to do studies with rewards if they succeed ... but that’s pretty close to wills proposal anyway.
Terrible idea: allow accelerated / less effort repurposing based trials of compounds that passed safety but not efficacy checks in past studies and then were not approved for efficacy
> But that requires a sort of covert acknowledgment that these drugs are probably being delayed too long
I doubt that when people support restrictions on drugs and stuff, they are thinking of *them* not being allowed to get something they want. They think of other irresponsible or stupid people. I can imagine supporting restrictions on an unsafe drug yet feeling it's an unfair privilege that someone specific gets to choose to circumvent them.
What they are imagining is taking an approved drug in good faith (because it was approved by the FDA and prescribed by a physician) and then having it turn out horribly -- the drug does something terrible that wasn't known ahead of time through carelessness, haste, political/economic influence, et cetera.
You are seeing that play out in real time in the reluctance some people have to get a COVID vaccine because of the speed with which it was approved. There are people who think there may be something screwy about the vaccine, like it has an 80% chance of giving you brain cancer 5 years after you take it, and nobody knows about it yet because 5 years haven't passed since it was invented.
And that is not a completely brain-dead attitude, either. We have plenty of examples of things that were at one time considered reasonable therapies or at least not harmful that proved to have disastrous long-term effects, e.g. smoking was once considered healthy, since it takes decades to develop lung cancer from it and it took people a good long time to realize that.
This attitude about vaccine harms down the line has made me wonder if a poison yet exist that could have serious side effects years down the line from one or two doses.
A sub immediately lethal dose of polonium would do that. Not much exists like that unrelated to oncogenesis (I mean, with critical development periods in pregnancy being one obvious exception).
Cassava contains cyanide. It does not kill you slowly. Cyanide is well excreted by the body. It has even been used at subtoxic doses recreationally(not recommended). Even long term continuous exposure from the environment does not kill you. It can however cause some health issues.
In fact, the process of cooking cassava (and several bean varieties) with the pot lid removed to expel cyanide as well, as washing used on nardoo, acorns and such, can never be 100% effective. That alone should indicate to you that there is not a long term buildup of these toxins.
Well, sure, e.g. the early pioneers in the discovery of nuclear radiation had no idea it could give you cancer decades later[1]. Lead is another poison the harm from which can manifest years to decades later.
"Poison" is a vague term, but prion diseases are an example of this sort of behavior. Exposure to a single small dose of prions causes no immediate effect, but severe to lethal consequences years later. It's obviously not a thing that happens frequently in drug or vaccine development, but I don't think we know enough about prions to absolutely rule it out.
If a doctor prescribes me a drug that I have to pay an arm and a leg for, I'd like to know it works, or at least that it wouldn't kill me. I don't have the time or expertise to hunt down and read scientific papers. The only way I'd know it works to a reasonable confidence and in reasonable time is if a government agency known for high standards approved it.
To be fair, I think they also think about vulnerable populations. If you release a new med, the side effects and negative interactions that end up killing people, will mostly kill people in vulnerable populations - the elderly, people on 10 different meds already, people with compromised health to begin with, etc.
If you're healthy already, you can be correct in thinking that the cost-benefit analysis for you is positive, but that it would be negative for enough people that making it universally available might be negative on the whole.
Even supposing Aducanumab doesn't treat Alzheimer's, if it truly is an anti-amyloid, it should have some beneficial effect. As you wrote earlier, doctors should be allowed to write prescriptions for it (since it seems safe), but insurance shouldn't have to pay for it for AD (since it doesn't seem effective against AD). Under our current system, a drug that might be useful against disease X will be banned forever if it's first tested against disease Y.
But the last ten anti amyloid drugs have not had and beneficial effects, and there’s a long history of misunderstandings or complications in mechanisms of drugs that “absolutely should be beneficial” not being
Technically, that is the situation now. Biogen can legally produce and market it as a remedy for Alzheimer's, and doctors can prescribe it for that condition (or anything else that crosses their minds), but insurance companies are generally under no obligation to pay for it. Nor does Medicare, until the CMS says it must.
What people point out, however, is that the political and market pressure to pay for it will be immense since the FDA approved it (and physicians and families are shortly to be subject to an onslaught of professionally-photographed heartstring-tugging marketing material from Biogen). Furthermore, the funding priorities of Big Pharma may shift, in the sense that the suits are always looking for an easier way to make the next $billion, and now that Biogen has paved a certain path, they may think following that path is easier than plowing another $250 million into Research Team X's kooky idea about what *else* might help with Alzheimer's, and so just possibly a far better therapy might be delayed considerably because it gets starved.
I don't have any brilliant solutions to the rat's nest of competing social, scientific, and economic forces here, but I do feel confident that anyone who fiercely argues the FDA is too damn strict and slow, and ought to let people try shit once it's been proved it isn't very dangerous and might or might not help cannot logically also argue that the FDA should never have approved aducanumab on the mere grounds that it hasn't been proved to help beyond any shadow of reasonable doubt. Those positions are logically incoherent.
Right. I should have written, "Under our current system, a drug that might be useful against disease X will probably remain banned forever if it's first tested against disease Y and fails FDA approval."
Maybe. I don't know, this passes my understanding of the field. I imagine it might depend on why it fails -- if it fails in Phase II (safety), yeah. But if it fails in Phase III (efficacy) I can readily see a big pharma company, after investing umpty $millions in the damn molecule, saying "well let's see if it works on anything else..."
That's right. The problem is that Medicare exists. It shouldn't. The FDA shouldn't stand in the way of anything that's safe. 3rd party payment is the problem.
Drugs are stupid expensive for 2 reasons. 1. Ridiculous barriers to entry
2. 3rd party payment, mostly medicare.
Hell, the copays for most doctors visits for people with insurance cost what the doctor's visit used to cost before HMOs and medicare.
What do you know. It turns out medicine wasnt the one area of the economy where throwing away market forces was a good idea.
Proving effectiveness (to FDA's high standards) doesn't become easier without Medicare. A lot of people here are concerned about the FDA making their lives worse (and sometimes blocking doctors from saving lives) by saying they can't even try a drug that might help.
Well...it would be hard to dump Medicare without a population willing to steel itself to old people getting no medical care, because there *will* be people who fail to properly prepare for old age, not to mention a modest number who did prepare but who suffered from bad luck (the insurance company folded, their investments tanked). Realistically, that's not going to happen.
What might work would be to reform Medicare so it only covers some basic level of care, with better care available through private add-on insurance (which already happens to some extent), and also make it an annual grant to seniors, with which which they can buy insurance on the open market, thus encouraging diversity (all the females can skip coverage for prostate cancer) and competition.
>>, if it truly is an anti-amyloid, it should have some beneficial effect.
No. This doesn't follow at all. 10 anti-amyloid therapies have failed before (I document this in the post Scott linked above). This one failed too. There is in fact /strong/ evidence against it working.
There is strong evidence against aducanumab /by itself/ working against a specific population (old people who already have pretty advanced AD, I assume, because that's what I always see in trials of AD drugs) of one specific neurodegenerative disease. There is stronger evidence that any curing or preventing of Alzheimer's, or of several other neurodegenerative diseases. will have to include a cure or prevention for amyloids.
You should read the Biogen trials. They're /very/ early-stage AD patient with MMSE of "24-30". MMSE scores of 24-30 are basically normal: 20-24 is mild dementia; 25-30 is a normal score.
The idea that the problem is trials just aren't targeting early enough AD patients is a common refrain but it isn't supported here.
>>"There is stronger evidence that any curing or preventing of Alzheimer's, or of several other neurodegenerative diseases. will have to include a cure or prevention for amyloids"
No. There is a wealth of animal data purporting to show this, but absolutely no human clinical trials show this. In fact, there is strong human data against it- large well-funded trials that reduce amyloid burden having absolutely no effect on cognition decline.
You're only looking at drug trials. See the comment below by "Aβ Male" for a summary of some of the evidence that amyloids are a problem. They are junk proteins that build up over time and eventually kill neurons. No one in the field questions this AFAIK. Anything that builds up over time and then kills neurons must be combatted, regardless of what drug trials (which are more complicated than establishing a mechanism, and can fail for many reasons unrelated to whether amyloids contribute to AD) show.
It certainly does, but bear in mind what you have are starting to have here is a competition between pet theories, viz. clearing amyloid will help, clearing amyloid definitely won't -- because the trial didn't (and can't) enroll enough people for long enough to fully test the hypothesis.
AD generally takes years to decades to manifest, and years to decades to kill. An 18-month trial with a modest number of participants and a particular measurement of results is not enough to rule out any benefit at all. It's quite fair to say the benefit has not been proved. It's also quite fair to say the benefit is not likely, based on what we know so far. But you can't say the benefit has been entirely ruled out on the basis of what we know so far. That's why the approval was difficult. That's why reasonable men can disagree on the best action.
We have empirical evidence that amyloids build up and kill neurons. Clinical trials are just one type of evidence, and they aren't designed to prove what you and Will are trying to use them to prove--in this case, that amyloid build-up isn't part of the mechanism behind aging. They are carefully designed only to evaluate whether a particular drug, administered in a particular manner, to a particular population that has developed a particular stage of a particular disease, will produce an effect with a p-value below a chosen cut-off.
In any case, all clinical trials are so similar in their approach that taking 10 clinical trials is not as powerful evidence as looking at 2 or 3 experiments which are truly independent. There's a large number of possible sources of error which most or all clinical trials are subject to, so all the clinical trials in the world still wouldn't provide two independent pieces of evidence.
Again, “drug targets mechanism correctly but it turns out we were wrong about mechanism, even if not fully wrong, enough that the way the drug targets it doesn’t actually help patients” is a somewhat common failure for drugs. “Amyloid important” and “drug remove amyloid” is possible without “drug good”. Why is the failure of all the past drugs in trials - drugs that successfully removed the protein stuff - and then the apparent failure of this one - enough to approve? Should the other ten antibodies will listed have been approved?
The $100 billion/year estimate from my blog post is a high end, and after writing that post, the FDA took a step back and restricted the indication to only early-stage patients. Still, it has the potential to be extremely costly.
Whenever you write about the FDA I can't help but think of the FAA. Drug companies have no incentive to sell dangerous drugs. Just like Boeing doesn't have an incentive to have planes suddenly plummet to the ground killing 100s. Boeing might not but individual Boeing employees certainly do. And not long after we started relaxing regulation those individual employees started to run amok. I don't see why we wouldn't' see the same issue with reduced regulation of Pfizer that we saw with Boeing.
There are a number of reports of safety issues in Boeing planes, with somewhat frequent mass groundings. I had read it had to do with organizational incentives and structure and culture and the ceo changing, idk how much about that is regulations.
The key issue was Boeing being allowed to certify itself. The theory being it would never let planes fall out of the sky. Just like the banks back in 2006 wouldn’t lend all their money to people with no ability to pay it back. Why would they do something so foolish?
“ The FAA delegated to Boeing itself most of the analysis and testing required to certify the MAX as safe to fly. … However, some Boeing ARs complained of heavy pressure from managers to limit safety testing and move quickly through the analyses. One AR who balked at such pressure was removed from the program.”
That's a nice little article. Two things jump out at me as significant factors contributing the problem:
(1) The reduction in authority of the actual engineers in favor of the technical managers, at both Boeing and the FAA.
(2) The decision of Boeing when the problem proved difficult ($$$) to fix by redoing the airframe to let the inherently dangerous behavior of the airframe stand, and "fix it in software" by creating a program to compensate.
Because both of these seem like general modern trends, and generally increasing risks. I can understand why both of them arise: engineering gets more and more complicated, and so people demand more people to whom they can talk in non-gobbledegook -- technical managers -- and they award those people more and more power. It works great in terms of nontechnical people being better able to *understand* what is going on, but it definitely increases the probability of Something Bad happening because the simplifications involved in the technical manager's understanding turned out to be *too* simplified.
The temptation to fix hardware problems in software is also easily understood, since software is cheap and fast and hardware is not, and evil combinations of flawed hardware and buggy software that are just right to cause disaster are ipso facto rare.
But still, they both seem like growing problems, and not easily solved, given the necessarily increasing distance between the frontier of science and technology and the technical competence the average citizen/consumer possesses.
>and evil combinations of flawed hardware and buggy software that are just right to cause disaster are ipso facto rare.
Not the first time it's happened; Therac-25 shot people with lethal radiation beams because of exactly this. Also, cars with auto-engaging out-of-lane prevention (which also prevents driving straight when there is reflective water or a not-in-program arrangement of lines on the road; "possessed by demons" indeed) - although I think that one happened after/around-the-same-time-as the MAX catastrophe.
Yes, and I expect it will happen more in the future. For a long time we kept software out of life-and-death situations, because it was so self-evidently risky. But we are much more accustomed to relying on software these days, and I think the pressure is rising to allow it into more serious situations, because of the immense savings in cost. I'm not sure we have the right set of attitudes and practices for that not to have some significant tragic outcomes, at least until we develop better attitudes and practices.
Ugh, yes. I recently had a Tesla Model S try to stuff me into the side of a tractor trailer on the Autobahn in a construction zone because the car got confused about which set of pavement markings was important. And then I had the passenger turn that shit *off* right fucking now.
When I bought my last car I was musing over some of these add-ons, automatic braking, lane-keeping and stuff, and the salesman (no spring chicken himself) looked at me and said "Dude...seriously, you've been driving 45 years, are you really going to trust the car to do your braking or steering for you?" and I realized he was 100% right. Maybe a person can develop new instincts of when to trust the car and when not to, but I'm too old for that crap. I don't even use the back-up camera much, I just turn and look like I always have, although I'm glad it exists to show me whether there's a cat or kid's tricycle right up against the bumper.
I didn't read that article, but isn't the main problem that the software lacked the redundancy it needed for safety (relying on one sensor instead of the usual standard of two sensors)? Combine that with the fact that pilots were unable to turn the system off (if they even knew it existed) and it was a recipe for disaster.
Depends who you ask. If you ask me, the main problem is the airframe was flawed, did dangerous stuff easily, and that they tried to fix it in software *at all*. Other people, who don't mind putting their lives in the hands of a computer program, will agree with you, and say the problem is that the program was not well designed, or integrated with hardware.
As I understand it, there's already a lot of safety-critical software controlling car behavior (remember all those Toyota unintended acceleration events?). I agree that in general that this is concerning because it's harder to achieve reliability in software than hardware. In this particular case, though, it's a huge error to force the nose down based on a single sensor reading, and this would still be true if it were a hardware circuit forcing the nose down. It's *so* obviously a bad idea that I've been hoping to hear a report explaining how the hell this could have happened without multiple key Boeing employees being grossly incompetent. Even without the regulator doing its job, a flaw this obvious shouldn't slip through.
The pilots were perfectly able to turn the system off. The penultimate Indonesian 737 MAX crew recognized that the automation was about to kill them, in a very slight variation of a common way for automation to kill people in airplanes, so they hit the kill switch that disabled that whole class of automation and flew the plane safely to its destination using the manual backup system. Which is why you never heard of them, or of countless other flight crews doing the same on many other airplanes most of which were not 737 MAXes.
The next Indonesian 737 MAX crew appear to have been clueless buffoons who should never have been allowed to fly an airliner, who in spite of the note from the last crew saying "PS automation is really flaky, don't trust it until it's checked out", never tried to switch to the manual backup.
The Ethiopian 737 MAX crew, appear to have properly recognized the problem, hit the kill switch, and gone over to manual backup controls. Then tried to selectively reactivate part of the automation, and been screwed over because Boeing changed the way the kill switch worked without documenting it. *That* was the unforgivable sin, IMO.
If the idea is that the automation must never ever ever ever fail, get real. Nobody has a clue how to make airliners cheap enough for non-rich you to afford a ticket, without some degree of automation. And nobody has a clue how to make airliners safe enough for you to tolerate them, without human pilots ready to lobotomize the robots and take control. So you make airplanes with lots of automation, because it's much more efficient when it works, and give the human pilots the tools and training and documentation they need to take over when the automation fails
I mean, the MCAS isn't exactly a normal autopilot. Its entire intent was to prevent pilots from inputting dangerous commands i.e. make an airframe safe that is *dangerous to manually pilot* (at least, without specific training, which Boeing wanted to avoid).
Using automation as a failsafe for an untrusted human pilot means you *can't* rely on the human pilot being a failsafe for the untrusted automation - as such, it should be held to a much higher standard than otherwise.
I believe you, but could you clarify something for me: if the pilot has to be there already to take over for the automation failing, why is it cheaper to automate? Can pilots log more hours when they let the plane fly itself as opposed to going manual the whole way? Or are we able to employ a less talented/educated (cheaper) pilot who only has to be capable in a small handful of "automation failed" scenarios and not in the wide away of "operate a plane normally" scenarios?
In the Indonesian case, wasn’t the aircraft also having numerous faults related to airspeed indicators starting on the takeoff roll (before MCAS kicked in)? Also, I thought the report indicated that the captain didn’t think to turn off the auto-trim, but did at least continually flick the trim switch in the yoke, which would override MCAS for a few seconds at a time. This kept them in the air for several minutes, until he handed control to the copilot - without telling him what he’d been doing to keep the aircraft controlled!
The reason Boeing was incentivized to cut corners with MCAS in the first place is that it would have been tremendously expensive to certify and deploy the MAX if they couldn’t pretend it was just a minor update to the existing 737.
The reason it would have been onerously expensive is because the FAA gives the FDA a real run for its money in making innovation insanely slow and costly.
FAA certification of a new aircraft type takes years and hundreds of millions of dollars. Pilots getting a “type rating” in a new aircraft costs the airline low 5 figures each initially, plus must be maintained through ongoing training and currency requirements for each type they fly. Having a mixed type fleet is costly - Southwest flies only 737s precisely for this reason. Type rating don’t really distinguish between “basically the same but for one or two small differences” and “completely different airframes”. A MAX under a new type would be just as different from the 737 NG as an ERJ or an A380.
So Boeing has turned the 737 into basically a ship of Theseus at this point, very little is unchanged since the original 60s design, but Boeing is heavily incentivized to make sure it can stay within the original type certificate, since otherwise they can’t be cost competitive. Airbus does exactly the same thing with the A320.
MCAS is probably not something Boeing would have implemented if the MAX was a clean sheet design (the airplane is basically fine without it, but it doesn’t fly just like an older 737). But the FAA’s onerous certification requirements pushed Boeing into design decisions that were made not for best engineering design, but to maintain apparent commonality.
It seems that the FAA mostly went along with this wink-nudge-its-just-a-737 thing because they fundamentally didn’t believe that treating it as if it was a brand new airplane either, but admitting reality would have forced them to do so, because the only thing that takes more time and money than certifying a new plane is changing any significant FAA rule, no matter how out of sync it is from modern reality.
This is an organization that took >20 years to approve a leaded gasoline replacement for general aviation (an industry where high FAA certification costs mean that basically everyone is flying around with 1950s engine technology)
I think the criticism raised by Charlie (and Kevin Drum, and others) just relies on... an unnatural reading of things? If you think a particular government agency should not exist -- if you think that the function it is supposed to perform is one that should not be performed, and therefore no agency should exist to perform it -- that would to my mind normally be described as you being "against" that government agency, and that is how I would normally someone interpret saying they are "against" a particular agency or that that agency is bad. I don't know why one would read it instead as a comment on the quality of the particular staff? I would normally think of government agencies primarily in terms of their function, not the particular people who happen to be executing that function...
Sure, but my comment generalizes to the case of "the function it is performing is the wrong function and therefore it should perform a different one". You don't have to intend to abolish the FDA to talk in terms of functions rather than the people tasked with those functions.
> But that requires a sort of covert acknowledgment that these drugs are probably being delayed too long - otherwise, these politicians are heroes for taking potentially-dangerous substances and testing them out for the rest of us.
I think there's an internally consistent position where the observer doesn't acknowledge that the drugs are being delayed too long, and just objects to the actor breaking the rules. Basically, in this framework whether the treatment works is moot, the objection is just about fairness and/or the morality of rule-following.
(Not supporting this position, in particular; just providing an observation for why most people don't think politicians are heroes when they cut the line, and also the criticism Biden is trying to pre-empt with his statement that he has to wait for FDA approval before taking his booster.)(
Andrew Gelman does a thing where he writes his content and puts them in a queue to post 6 months-ish later. Maybe that's an option foryour less time sensitive pieces.
Sometimes you can understand an acronym even without having it explained (well, at least I can). For example, if you already know "IIRC" ("if I recall correctly"), you can probably understand the sentence "IIUC, photosynthesis converts light energy into chemical energy".
Point. (Though I'm not sure I would have gotten that particular example if you hadn't primed me first.)
But that particular strategy only seems like it would work for stock phrases your audience already expects you to say based on context. If you are trying to promote an idea that is not already widely-talked about, I'm not sure it can help you.
I think he meant 'explain' as in 'explain the joke and how it is supposed to relate to the situation at hand', not just 'say what the acronym stands for.'
The final example about bupropion-dextromethorphan highlights how misguided it is to solely fund clinical trials through the promise of patent-protection. Presumably the government could make a guess at how likely this would be to work, how valuable it would be if it did, and then the government could have people bid on the contract to run a clinical trial.
Does this happen? Does the government, or anyone, fund clinical trials with the condition that anyone can make the drug if it is approved? The Ketamine example fits here too it seems.
Funding agencies and hospitals will occasionally run trials with generic drugs or to figure out specific clinical questions with already approved drugs-- how long should antibiotic X be given? 10 vs 8 days. Etc.
I don't think this happens nearly enough, however.
I'm kind of baffled how you can invoke the "normal knowledge that normal people use" as some kind of accessible source of information when nobody has the first clue how to gather or assess that. Normally among pundits it operationally reduces to "what me and the people I know/read/like think" but that leads straight to the Pauline Kael effect, and some of the unpleasant divisions between "expert opinion" and what vast swathes of regular folks think that we see today on any number of issues.
If nothing else, consider that the "knowledge of normal people" about which we have cared deeply for several centuries is "who should run the country?" and for this purpose we have this vast apparatus of elections, at various levels, terms of various sizes, representation or not, not to mention polling and focus groups and prognostication -- and yet we *still* are routinely surprised by the outcomes, and debate furiously just how genuinely representative they are.
What hope is there of actually discovering what the "knowledge of normal people" actually is on the question of whether Drug X should/should not have been approved, or whether the FDA should be "stricter" or "looser" in general, for various definitions of "strict" or "loose?"
I feel like all you're really saying that's 100% defensible here is that given time, a consensus will emerge, and using the clarity of hindsight we will all be able to say decision Z or T of the FDA was right/wrong or hasty/slow. But noting that hindsight is easier and more accurate than foresight is rather a truism and (barring the invention of time machines) gives no clue on how to improve the situation going forward.
Scott's position is that the consensus often emerges before the FDA makes a decision.
This is not a mechanism that Scott is proposing, it is a standard. If everyone (in the medical community?) already knows that e.g. covid vaccines will be approved for children and the FDA hasn't allowed it yet, then the FDA is moving too slow.
Yes, well, the gist of what I said is that I have very strong a priori doubts that this is true -- that one can empirically measure a reliable "consensus" (among whom? would be the first question) before the FDA makes a decision.
As a rule, people react to events, they do not work proactively to form an opinion on events that have not yet happened yet. Exempli gratia, I doubt except among those most intimately concerned (patients, physicians, researchers) anyone even tried to form an opinion on whether aducanumab should be approved in advance of the event.
And if the "consensus" which we intend to consult is merely among the small community who are intimately engaged ahead of time, because of assorted intersts, but...er...not the *same* small community that is intimately engaged in FDA approval, this (1) does not meet the definition of some broad-based "general commen-sense person" community, and (2) isn't obviously better than what we already have.
Perhaps I should add that I don't doubt a consensus may emerge in some random community or other -- e.g. "all the people I know" -- but as I said at the top, that is a very distinct thing from a general broad-based consensus, and the former should not be mistaken for the latter.
I think the general situation can be 'incentive hacked' from above in a way that simply pressures the experts in the correct direction
Give everyone at the FDA a 10% raise and then make 20% of their paycheck dependent on their performance in terms of a chosen cost to benefit or cautionary principle standard. Something like "10 people's lives are not saved due to overcaution so that 1 person's life is not lost due to lack of caution"
If the FDA is responding to huge overcaution incentives where nobody notices if they don't approve something and good things do not happen and someone loses their job if they do approve something that goes wrong then the large paycheck incentive may fix that and at least might serve as a red flag that the process has problems
At the same time if the FDA is not internally able to approach the quantified goal they themselves will be the ones waving the red flag and identifying what the obstacles are
Who manages the managers? Who watches the watchmen? This just declares an intent to fix things and says “someone’s job will be to make the correct decisions, somehow? And they’ll be responsible for the actual decisions”. It doesn’t fix either the “what is the actual principles we can teach and carry for what to approve and how and when”, and “how does your incentive system not bend when Eric topol says to not rush the vaccine or your expanded approval system (whether or not it really did, doesn’t matter, just the pressure) kills ten black babies in underprivileged communities.”
It would be straightforward enough to be able to assess whether the performance measurement of the FDA was reasonable. We all know that pausing J&J may have saved a few lives and definitely cost, what, thousands? That one at least is pretty bright and easy
One thing the incentive pressure would likely immediately do is cause the FDA to choose a number of other regulatory agencies like the EU's and provisionally or fully approve everything they do because the likelihood is they are overcautious as well. Suddenly the regulatory problem is reduced to whichever agency is the best, assuming the primary problem is in fact overcaution. (I take this almost for granted given the incentives that are at play but I understand that emotionally there is a constituency for allowing 100 or 1000 lives not to be saved so one does not die from undercaution because for some it does feel more wrong for someone to die as opposed to 100 not being saved. What the FDA might do under pressure from a "ok, but here's how far you can go with that" incentive is open up provisional approval that allows people to use drugs that are not fully approved under their own volition and request that the calculation of their performance on not approving that drug yet be suspended since people have at least been given the choice to take their own risk)
Re: your second response below I'd say 20% of someone's paycheck is enough to become the primary incentive in their work. If it doesn't we should hear about just what the problem is because there will at least be an incentive to complain
> It would be straightforward enough to be able to assess whether the performance measurement of the FDA was reasonable
Not really? That sounds somewhat hard. Let’s say the FDA approved all ten Alzheimer’s antibodies right now, but didn’t mandate insurance prescription. And then let’s say they opened a rolling trial option where up to 10k total patients per drug may pay up to $100 each to participate in a trial for drugs that have passed animal safety checks and human safety model checks, to speed up development and use that data for phase three trials, and then this is used for more monoclonal antibodies and also for early likely to fail 3D printed organ test transplants in terminal patients who will die anyway to speed up testing. Should that be rated as Good or Bad by your agency? Is it obvious? Not really
I don’t like the idea of companies regulatory shopping for which of seven agencies with partial mutual recognition of approval is the most generous or corruptible. In some areas that’s fine, but for this one eh. I’d be happy with re using trials and evidence used for other countries if they pass our standards but I’m pretty sure we already do that
20%+ paychecks are not enough to be a primary incentive if, for instance, you will have your agency power taken away if you do the thing, by public backlash. Might also be thinking more of a corporation and less about the actual structures at play in the FDA - who’s making the decisions? Who’s salary so you increase? The political appointed head? Political appointees of all people will have other incentives, and potentially limited ability to fight the careerists if they even want to.
I think 'voting is wonky so we can't learn public opinion about anything' is a bad take.
First of all, in our two-party system, the parties autoamatically re-align themselves so that each side is always capturing about 50% of the population, making the decisions close and hard to call all the time, and there are massive incentives on all sides both to pervert the process of finding out what people want and to misinform people about what they should want. This is probably the hardest version of this problem imagineable, because of the unique circumstances surrounding it.
If you wanted to find out if, say, the majority of people believe that rain is cause by pixies crying when they see a clown get hurt, it would not be at all difficult to get this information. You would do a phone survey with a few hundred or thousand respondents, the outcome wouldn't even be close, and the percentages you get would be very close to accurate.
Those are the two extremes of the easiest possible problem vs the hardest possible problem, but I think that most things you might care to ask about are closer to the crying pixie end than to the voting for president end of the spectrum.
I think it *feels like* the population is really divided on most issues and it's hard to ind out what they really think, precisely because we spend all of our time talking about controversial issues that are tied up with politics and active manipulation. But the vast majority of issues you might ask someone about are not controversial in this way, and on those I think it's not that hard to get a public consensus.
I certainly agree on many issues people *can* be assessed for their understanding and viewpoint pretty easily. But not in the slightest with respect to medicine. People feel *way* more strongly and emotionally about personal health than they do even about who is President. You can take the most rabid Trump-really-won fanatic and the die-hard AOC fanboi and all their political differences will vanish like a pricked balloon if the question before them is how to prevent themselves, a spouse, or a child from snuffing it from a brain tumor, and whether Drug X will do that or not, and whether it should be available, cost what it does, et cetera. Health care is *even more* screwed up with passions and illogic than politics.
""On March 19, 1972, the FDA reassigned Nestor to its Office of Compliance, as part of a reorganization in which Civil Service doctors were replaced as drug approvers by consultants who were often affiliated with commercial companies. Nestor began a grievance proceeding. A review panel determined that senior FDA officials gave "misleading" testimony against Nestor and concluded that Nestor was due an apology and appropriate duties.""
The final decision is made by a bunch of men-on-the-street: a jury. We decided that, even though justice system officials know a lot more about crime, it is too difficult to get rid of their inherent biases, so a group of men-on-the-street does better.
If we take this analogy even further, we could have juries for the FDA. The experts can make their arguments, but the decision would ultimately be decided by a group of men-on-the-street. Random people would get called to make decisions that effect the life and death of their fellow citizens.
While I think that this is interesting, I'm not convinced that it is a good idea. The FDA is often too slow to approve good drugs and the jury system is not exactly known for its speed.
Judges make plenty of decisions in criminal cases, and juries are notorious for not understanding and making somewhat mistaken decisions in trials involving detailed and complex technical evidence or issues. (Judges also do that, but the FDA’s experts are subject matter experts)
I don't think the analogy is especially applicable. The role of a jury is a trier-of-fact. That is, their job is to assess the believability of the witnesses both sides put on, and decide who to believe. The *judge* is the one who decides what the law says, i.e. given this pattern of facts on which the jury agrees, what is the crime (if any) that was committed, and what is the appropriate result?
There's a very decent reason to believe that a panel of average strangers are well positioned to collectively decide whether Witness X is telling the truth or lying. It's a generally-distributed skill, for which there is no known education or technology that could do a better job, it's a collective decision, so individual flakiness is averaged out, and the people involved are very detached from the rest of the judicial process, so they are not influenced by considerations of the sentence, of the effect on other defendants or cases, of the nature of the law, et cetera -- all of which can have powerful effects on lawyers, prosecutors, and judges, the experts in the bowels of the system.
There isn't much that's analogous in the FDA's decisions. A major part of what a criminal trial is about is deciding who's lying, and there is (one hopes!) essentially none of that in the FDA's decision process.
Furthermore, the assumption that would seem to underly this, that an *individual* man on the street has a better judgment of the details than an individual FDA apparatchik, seems dubious a priori and anyway isn't part of Scott's proposal -- I believe he's arguing that some vast consensus, the wisdom of the crowds, so to speak, can exceed the good sense of a much smaller number of FDA experts and bureaucrats.
That is, the key change is to make the number of people who influence the decision much, much bigger. (This may also be the assumption underlying the idea of taking guidance from approvals in other countries -- basically, if the EU and UK and Japan all approve a drug faster than the FDA, maybe the wisdom of the crowds is telling us something.)
Honestly, I think juries are in the wrong place. I'd rather put the "committee of men-on-the-street" as one of the houses of parliament (Congress in the US), where they can tear up criminal overreach at the source.
Does the USA Health system have an equivalent of the UK's NICE ? If only for the government-funded parts of the system?
The National Institute for Co-ordinated Experiments? I hope not. ;)
Huzzah for the Lewis reference.
That has been proposed at various points but people called them death panels.
Thank God no. The solution to a clumsy but powerful bureaucracy is not to create an entirely new level of even more powerful bureaucracy and hope that, against all experience, it will be sleek, intelligent, and efficient.
There's a child's rhyme about this, about an old lady who swallowed a spider to catch a fly.
Whilst I agree in principle here, what is being advocated in effect is having a government-funded scheme with no mechanism for controlling costs. Or to put it another way, a way for the cartel of approved manufacturers to take money from tax payers. You need to have a check on that process even if it is additional bureaucracy as the costs are going to be less. Ideally though I guess the check should be democratic rather than bureaucratic.
Yes. The only effective restraint on government is active and informed voters. Asking government to keep an eye on government is deeply silly, and never works. I will say a kind word for federalism, though -- having roughly equipotent goverments keeping an eye on each other has some real value, or used to, back when federalism was taken more seriously. It also gives citizens the ability to vote with their feet.
I'm British. I think we're largely happy with NICE, both the population in general and the doctors. They are slow to react, but the general principle of 'government only pays for cost-effective treatment and cost-effectiveness means price less than "£X/QALY" seems pretty sound.
If you guys had the FDA to say "safe enough to be used", and a NICE equivalent to say "useful enough to pay for centrally", your above problem would be solved.
Glad you like your system. I wouldn't have it if you paid me. The only person I want to decide whether a given drug is useful to me or not is me, and I'm quite happy to pay for it, or buy insurance in advance to cover its cost, in order to have that right. I'm gobsmacked the Brits, historically pretty stout defenders of their personal liberty, are willing to turn over that much authority to Parlaiment. Why not have them arrange your marriages while you're at it, eh?
What system does your insurance company use to decide something is "worth it"? Do they do all their own $/QALY calculations each day? Can you compare one insurance company's "worth it" against another's "worth it" to decide which is more in line with your values?
NICE only determines what standard NHS care will cover.
"I'm quite happy to pay for it, or buy insurance in advance to cover its cost, in order to have that right" would absolutely still be an option.
People often seem to act like the existence of the NHS somehow means that private care and private health insurance don't exist over here, and it simply isn't true (I think Canada is closer to the strawman I keep seeing people attack?) - a more accurate model would be to consider the NHS as Universal Basic Insurance.
Briefly thought Dave Barry was posting on ACX. :(
Dansuul says:
"a scientific organization (which the FDA is)"
Is that the primary goal, though? To do "Science"? Seems to me like it has a relatively large "cost-benefit analysis" role as well, which, as the voting citizens of the country that created the FDA, and who will bear both the costs and benefits of those analyses, I'm not sure it's the "man on the street" unjustly intruding on the FDA's "scientific" jurisdiction. It seems more like the FDA unjustly (and without the relevant expertise) intruding on citizens' implicit demand that their public health agencies serve their best (again, implicit in "best" is some kind of cost/benefit analysis) interests.
Scientific cost benefit analysis is still science, which is a very broad and inclusive term.
Not unless it includes some experiment, and I'm hard-pressed to see what that experiment would be. As an actual real-life scientist, I'm routinely annoyed to see almost anything that involves numerical calculation and/or logic called "science" on the grounds that this is what "science" is. That's mistaking the tools for the task. It's as absurd as saying algebra can be classified as a form of poetry, like Shakespeare, because it (algebra) involves the meaningful arrangement of letters of the English alphabet.
Nothing is a science unless it begins with discovery through experiment. Anything else might reasonably be called philosophy, even natural philosophy, but if there isn't an experiment at the base, it's not science. And for that matter, the use of mathematics is not required: if I hypothesize that it rains every other Tuesday, and conduct the experiment of going outside and returning to drip, or not, on the calendar, I am doing science and my results -- the calendar with rain spots on certain days -- are scientific results, despite the utter absence of math.
</rant>
I think it’s complicated though. The FDA regulates the trials necessary for scientific investigation of drugs, and in that sense very much participates in the process of drug discovery. They also require complex opinions and decision making from subject matter experts in the scientific fields. So they’re not really “doing science” in the creative sense, but they employ scientists to direct and regulate the process of science (of evaluating and using drugs and other medical interventions). Science in genera is a slippery word and idea anyway
OP also said they lack “the relevant expertise” and I don’t think that’s true - more that they’re misusing it
I agree with your below comment - they still require (or should) all the scientific expertise that the real scientists need to do their work
Sure, they employ scientists, because they need to understand the science, but understanding the science isn't doing science, any more than appreciating poetry is the same as writing the stuff.
I don't think "science" is inherently a slippery word, but a great number of people and interests have made it so because they want to borrow some of the gloss that empirical science in narrow fields has built up for itself over the last few centuries. It's a common form of linguistic corruption.
Perhaps in some general sense, but in the context of this comment about the FDA, it is not the "science" that they are doing - and it is being explicitly contrasted against the "man on the street's" non-scientific "cost/benefit analysis" approach. "Science" defined so broadly that it includes the explicitly political (in a good way), cost/benefit analysis of "gee, maybe we should accelerate approval of a vaccine, given that thousands are dying each day" is meaningless - and worse, if it is "science," then it means that the FDA is not a scientific organization because it's obvious they aren't doing this - by their own admission.
The FDA is not even remotely a scientific organization. It's a regulatory standards-setting body, somewhat akin to NIST or the standards-setting part of the NHTSA or FAA (i.e. without the investigations arm). They are supposed to assess the evidence given to them about new drug efficacy and side-effects, and decide whether to approve or not the proposed use of the drug. Not only do they not do original science, they are not even like a scientific management committee, because they don't propose new areas of research, they don't favor or disfavor this or that line of research, or this or that research spending priority -- they just look at the applications laid before them and make a judgment on it.
I guess you can call them "scientific" in the sense that theyr'e supposed to (1) understand the science they're called upon to judge, so they need to have a background understanding of science, and (2) they are supposed to make their decisions based on a rational assessment of the measured effects -- and not on political, economic, or philosophical grounds.
This is actually not true. The FDA does have a few smallish groups that engage in actual scientific research, often in collaboration with biopharmaceuticals companies. These groups are kept separate from the regulatory function, but I imagine they likely still interact with the regulators. The FDA also hosts seminars delivered by biopharma experts on emerging technologies. Generally, I've found the FDA folks to be quite scientifically knowledgeable, at least within their own specialty. The FDA inspectors are particularly good; they are both feared and respected in the sense that they are quite good at finding legitimate deficiencies.
My point here, the problems at the FDA don't stem from incompetence or poor scientific knowledge. They stem from bad incentives and a flawed system.
Source: I am a co-author on a paper generated from an FDA/biopharma collaboration, and have been involved with plenty of FDA audits.
I think perhaps what you want to say is "that's true in a general way, but there are interesting exceptions -- they have some small groups that actually participate in research. Plus they sponsor seminars and such." That's a very valuable addition, and I thank you for adding it. But I still think a strong general statement needs to be made, because I suspect there are plenty of regular folks who actually think the FDA directs research, and the fact that it doesn't is important.
Thanks for linking to my piece. An interesting problem you bring up in the Axsome story is effectively a public goods provisioning problem: how do we incentivize good trials for generic drugs for repurposing?
Good trials for generic drugs provide the public good of accurate information on a clinical question, and then anyone can use that information.
Some ideas to make this better, ignoring novel drugs to simplify:
1. making clinical trials cheaper per patient. I don't know enough about this to have specific proposals, but it is something I'm eventually going to learn more about. Tentatively: something involving substantial reductions in the power of IRB's and using tech to make multi-site trials easier.
2. prizes for researchers who successfully discover a new use for a generic drug and prove it with a trial. This exists informally, in that a highly cited research trial paper will give you career capital and speaking invites.
3. Something involved prediction markets and clinical trial outcomes? Maybe people who are good predictors of successful drug repurposing should get $$ to choose more drug repurposing candidates?
I like your suggestions, especially the prizes.
I've always liked the idea (using the tiered/unbundled FDA Scott describes) + "human challenge trials", that after we do normal trials to establish initial safety (i.e. you don't die when you take it) then we open it up to a "soft approval" where you can legally take it, but you have to sign a really big waiver saying "I KNOW THIS ISN'T FDA APPROVED AND I WON'T SUE" and also "I PROMISE TO BE A PARTICIPANT IN THE ONGOING TRIALS FOR EFFICACY" and then also pay people enough (maybe make Prestigious Advancer of Science Through Clinical Trials a viable career option) to participate.
For 3, part of the problem is that one can’t really make as much money off of generics? Maybe a smaller but significant monetary reward for the company/individuals that pushed through the trial (like 2), maybe per drug prescription, as that more directly replicated the incentive for new drugs and provides the money right to the people who develop them.
I’m not sure the scale works there, you can’t really generate the same amount of money that a very expensive new drug candidate that isn’t genetic would sell, so it’s still very lopsided. I think 1) is the best option, but could mean many different things - it’d be really interesting if a group of doctors could run a multi center trial “themselves” without needing significant funding or backing.
Or you could set up a publix, taxpayer-funded body to do it. Market mechanisms are good at solving a lot of problems but not everything is a nail.
A public body to ... study repurposed generics? It seems like the sort of thing where the problem space (all diseases) times the solution space (all existing drugs) times the space of domain experts (for specific techniques and diseases and drugs and whatever) times people who have the disease and are willing to take it times the people who can administer and run the trial ... is very large, and would be better with some mechanism where more people and companies can participate if they think they have something good. So you’d want big biotechs and small biotechs and individual doctors or researchers and universities and whatever to all be able to do (I don’t know how you both make it compensated and doable at the high standard of a clinical trial but it’s worth thinking about) the sort of repurposing research here. Or maybe federal funding for research in it like public research, paying people to do studies with rewards if they succeed ... but that’s pretty close to wills proposal anyway.
Terrible idea: allow accelerated / less effort repurposing based trials of compounds that passed safety but not efficacy checks in past studies and then were not approved for efficacy
> But that requires a sort of covert acknowledgment that these drugs are probably being delayed too long
I doubt that when people support restrictions on drugs and stuff, they are thinking of *them* not being allowed to get something they want. They think of other irresponsible or stupid people. I can imagine supporting restrictions on an unsafe drug yet feeling it's an unfair privilege that someone specific gets to choose to circumvent them.
What they are imagining is taking an approved drug in good faith (because it was approved by the FDA and prescribed by a physician) and then having it turn out horribly -- the drug does something terrible that wasn't known ahead of time through carelessness, haste, political/economic influence, et cetera.
You are seeing that play out in real time in the reluctance some people have to get a COVID vaccine because of the speed with which it was approved. There are people who think there may be something screwy about the vaccine, like it has an 80% chance of giving you brain cancer 5 years after you take it, and nobody knows about it yet because 5 years haven't passed since it was invented.
And that is not a completely brain-dead attitude, either. We have plenty of examples of things that were at one time considered reasonable therapies or at least not harmful that proved to have disastrous long-term effects, e.g. smoking was once considered healthy, since it takes decades to develop lung cancer from it and it took people a good long time to realize that.
This attitude about vaccine harms down the line has made me wonder if a poison yet exist that could have serious side effects years down the line from one or two doses.
A sub immediately lethal dose of polonium would do that. Not much exists like that unrelated to oncogenesis (I mean, with critical development periods in pregnancy being one obvious exception).
Improperly prepared foodstuffs. Cassava is the main one here https://en.wikipedia.org/wiki/Cassava#Food_use but there are other food sources, such as nardoo, which if you don't prepare it properly will slowly to rapidly kill you: https://en.wikipedia.org/wiki/Marsilea_drummondii
But I don't think eating them once or twice would hurt you much.
Cassava contains cyanide. It does not kill you slowly. Cyanide is well excreted by the body. It has even been used at subtoxic doses recreationally(not recommended). Even long term continuous exposure from the environment does not kill you. It can however cause some health issues.
In fact, the process of cooking cassava (and several bean varieties) with the pot lid removed to expel cyanide as well, as washing used on nardoo, acorns and such, can never be 100% effective. That alone should indicate to you that there is not a long term buildup of these toxins.
Well, sure, e.g. the early pioneers in the discovery of nuclear radiation had no idea it could give you cancer decades later[1]. Lead is another poison the harm from which can manifest years to decades later.
-------------
[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3520298/
"Poison" is a vague term, but prion diseases are an example of this sort of behavior. Exposure to a single small dose of prions causes no immediate effect, but severe to lethal consequences years later. It's obviously not a thing that happens frequently in drug or vaccine development, but I don't think we know enough about prions to absolutely rule it out.
_Great_ example! They're kind of like a biological equivalent of ice nine from Cat's Cradle.
If a doctor prescribes me a drug that I have to pay an arm and a leg for, I'd like to know it works, or at least that it wouldn't kill me. I don't have the time or expertise to hunt down and read scientific papers. The only way I'd know it works to a reasonable confidence and in reasonable time is if a government agency known for high standards approved it.
It doesn't seem like that would require the substances that haven't gotten that seal of approval to be illegal, though?
To be fair, I think they also think about vulnerable populations. If you release a new med, the side effects and negative interactions that end up killing people, will mostly kill people in vulnerable populations - the elderly, people on 10 different meds already, people with compromised health to begin with, etc.
If you're healthy already, you can be correct in thinking that the cost-benefit analysis for you is positive, but that it would be negative for enough people that making it universally available might be negative on the whole.
(I may have expressed myself very badly given the replies I got)
Even supposing Aducanumab doesn't treat Alzheimer's, if it truly is an anti-amyloid, it should have some beneficial effect. As you wrote earlier, doctors should be allowed to write prescriptions for it (since it seems safe), but insurance shouldn't have to pay for it for AD (since it doesn't seem effective against AD). Under our current system, a drug that might be useful against disease X will be banned forever if it's first tested against disease Y.
"Under our current system, a drug that might be useful against disease X will be banned forever if it's first tested against disease Y."
That's not true at all.
https://ashpublications.org/blood/article/111/8/3968/24563/Thalidomide-for-treatment-of-multiple-myeloma-10
It's mostly true, because it's what usually happens.
But the last ten anti amyloid drugs have not had and beneficial effects, and there’s a long history of misunderstandings or complications in mechanisms of drugs that “absolutely should be beneficial” not being
Technically, that is the situation now. Biogen can legally produce and market it as a remedy for Alzheimer's, and doctors can prescribe it for that condition (or anything else that crosses their minds), but insurance companies are generally under no obligation to pay for it. Nor does Medicare, until the CMS says it must.
What people point out, however, is that the political and market pressure to pay for it will be immense since the FDA approved it (and physicians and families are shortly to be subject to an onslaught of professionally-photographed heartstring-tugging marketing material from Biogen). Furthermore, the funding priorities of Big Pharma may shift, in the sense that the suits are always looking for an easier way to make the next $billion, and now that Biogen has paved a certain path, they may think following that path is easier than plowing another $250 million into Research Team X's kooky idea about what *else* might help with Alzheimer's, and so just possibly a far better therapy might be delayed considerably because it gets starved.
I don't have any brilliant solutions to the rat's nest of competing social, scientific, and economic forces here, but I do feel confident that anyone who fiercely argues the FDA is too damn strict and slow, and ought to let people try shit once it's been proved it isn't very dangerous and might or might not help cannot logically also argue that the FDA should never have approved aducanumab on the mere grounds that it hasn't been proved to help beyond any shadow of reasonable doubt. Those positions are logically incoherent.
Right. I should have written, "Under our current system, a drug that might be useful against disease X will probably remain banned forever if it's first tested against disease Y and fails FDA approval."
Maybe. I don't know, this passes my understanding of the field. I imagine it might depend on why it fails -- if it fails in Phase II (safety), yeah. But if it fails in Phase III (efficacy) I can readily see a big pharma company, after investing umpty $millions in the damn molecule, saying "well let's see if it works on anything else..."
That's right. The problem is that Medicare exists. It shouldn't. The FDA shouldn't stand in the way of anything that's safe. 3rd party payment is the problem.
Drugs are stupid expensive for 2 reasons. 1. Ridiculous barriers to entry
2. 3rd party payment, mostly medicare.
Hell, the copays for most doctors visits for people with insurance cost what the doctor's visit used to cost before HMOs and medicare.
What do you know. It turns out medicine wasnt the one area of the economy where throwing away market forces was a good idea.
It's nice to see other people expressing this position. I often feel very lonely here in "perhaps we should try actual free market forces" land.
Proving effectiveness (to FDA's high standards) doesn't become easier without Medicare. A lot of people here are concerned about the FDA making their lives worse (and sometimes blocking doctors from saving lives) by saying they can't even try a drug that might help.
Well...it would be hard to dump Medicare without a population willing to steel itself to old people getting no medical care, because there *will* be people who fail to properly prepare for old age, not to mention a modest number who did prepare but who suffered from bad luck (the insurance company folded, their investments tanked). Realistically, that's not going to happen.
What might work would be to reform Medicare so it only covers some basic level of care, with better care available through private add-on insurance (which already happens to some extent), and also make it an annual grant to seniors, with which which they can buy insurance on the open market, thus encouraging diversity (all the females can skip coverage for prostate cancer) and competition.
>>, if it truly is an anti-amyloid, it should have some beneficial effect.
No. This doesn't follow at all. 10 anti-amyloid therapies have failed before (I document this in the post Scott linked above). This one failed too. There is in fact /strong/ evidence against it working.
There is strong evidence against aducanumab /by itself/ working against a specific population (old people who already have pretty advanced AD, I assume, because that's what I always see in trials of AD drugs) of one specific neurodegenerative disease. There is stronger evidence that any curing or preventing of Alzheimer's, or of several other neurodegenerative diseases. will have to include a cure or prevention for amyloids.
You should read the Biogen trials. They're /very/ early-stage AD patient with MMSE of "24-30". MMSE scores of 24-30 are basically normal: 20-24 is mild dementia; 25-30 is a normal score.
https://investors.biogen.com/static-files/8e58afa4-ba37-4250-9a78-2ecfb63b1dcb
The idea that the problem is trials just aren't targeting early enough AD patients is a common refrain but it isn't supported here.
>>"There is stronger evidence that any curing or preventing of Alzheimer's, or of several other neurodegenerative diseases. will have to include a cure or prevention for amyloids"
No. There is a wealth of animal data purporting to show this, but absolutely no human clinical trials show this. In fact, there is strong human data against it- large well-funded trials that reduce amyloid burden having absolutely no effect on cognition decline.
You're only looking at drug trials. See the comment below by "Aβ Male" for a summary of some of the evidence that amyloids are a problem. They are junk proteins that build up over time and eventually kill neurons. No one in the field questions this AFAIK. Anything that builds up over time and then kills neurons must be combatted, regardless of what drug trials (which are more complicated than establishing a mechanism, and can fail for many reasons unrelated to whether amyloids contribute to AD) show.
No. Empiricism beats pet theories 100% of the time
It certainly does, but bear in mind what you have are starting to have here is a competition between pet theories, viz. clearing amyloid will help, clearing amyloid definitely won't -- because the trial didn't (and can't) enroll enough people for long enough to fully test the hypothesis.
AD generally takes years to decades to manifest, and years to decades to kill. An 18-month trial with a modest number of participants and a particular measurement of results is not enough to rule out any benefit at all. It's quite fair to say the benefit has not been proved. It's also quite fair to say the benefit is not likely, based on what we know so far. But you can't say the benefit has been entirely ruled out on the basis of what we know so far. That's why the approval was difficult. That's why reasonable men can disagree on the best action.
We have empirical evidence that amyloids build up and kill neurons. Clinical trials are just one type of evidence, and they aren't designed to prove what you and Will are trying to use them to prove--in this case, that amyloid build-up isn't part of the mechanism behind aging. They are carefully designed only to evaluate whether a particular drug, administered in a particular manner, to a particular population that has developed a particular stage of a particular disease, will produce an effect with a p-value below a chosen cut-off.
In any case, all clinical trials are so similar in their approach that taking 10 clinical trials is not as powerful evidence as looking at 2 or 3 experiments which are truly independent. There's a large number of possible sources of error which most or all clinical trials are subject to, so all the clinical trials in the world still wouldn't provide two independent pieces of evidence.
Again, “drug targets mechanism correctly but it turns out we were wrong about mechanism, even if not fully wrong, enough that the way the drug targets it doesn’t actually help patients” is a somewhat common failure for drugs. “Amyloid important” and “drug remove amyloid” is possible without “drug good”. Why is the failure of all the past drugs in trials - drugs that successfully removed the protein stuff - and then the apparent failure of this one - enough to approve? Should the other ten antibodies will listed have been approved?
The $100 billion/year estimate from my blog post is a high end, and after writing that post, the FDA took a step back and restricted the indication to only early-stage patients. Still, it has the potential to be extremely costly.
Whenever you write about the FDA I can't help but think of the FAA. Drug companies have no incentive to sell dangerous drugs. Just like Boeing doesn't have an incentive to have planes suddenly plummet to the ground killing 100s. Boeing might not but individual Boeing employees certainly do. And not long after we started relaxing regulation those individual employees started to run amok. I don't see why we wouldn't' see the same issue with reduced regulation of Pfizer that we saw with Boeing.
Can you cite some particular examples of Boeing employees running amok after the regulations relaxed?
Airplanes are the safest form of transportation [1], so it seems weird to me that you're using them as an example of excessively lenient regulation.
[1] https://www.washingtonpost.com/news/wonk/wp/2015/05/14/the-safest-and-deadliest-ways-to-travel/
There are a number of reports of safety issues in Boeing planes, with somewhat frequent mass groundings. I had read it had to do with organizational incentives and structure and culture and the ceo changing, idk how much about that is regulations.
The key issue was Boeing being allowed to certify itself. The theory being it would never let planes fall out of the sky. Just like the banks back in 2006 wouldn’t lend all their money to people with no ability to pay it back. Why would they do something so foolish?
Government bailouts?
“ The FAA delegated to Boeing itself most of the analysis and testing required to certify the MAX as safe to fly. … However, some Boeing ARs complained of heavy pressure from managers to limit safety testing and move quickly through the analyses. One AR who balked at such pressure was removed from the program.”
https://www.seattletimes.com/business/boeing-aerospace/what-led-to-boeings-737-max-crisis-a-qa/
That's a nice little article. Two things jump out at me as significant factors contributing the problem:
(1) The reduction in authority of the actual engineers in favor of the technical managers, at both Boeing and the FAA.
(2) The decision of Boeing when the problem proved difficult ($$$) to fix by redoing the airframe to let the inherently dangerous behavior of the airframe stand, and "fix it in software" by creating a program to compensate.
Because both of these seem like general modern trends, and generally increasing risks. I can understand why both of them arise: engineering gets more and more complicated, and so people demand more people to whom they can talk in non-gobbledegook -- technical managers -- and they award those people more and more power. It works great in terms of nontechnical people being better able to *understand* what is going on, but it definitely increases the probability of Something Bad happening because the simplifications involved in the technical manager's understanding turned out to be *too* simplified.
The temptation to fix hardware problems in software is also easily understood, since software is cheap and fast and hardware is not, and evil combinations of flawed hardware and buggy software that are just right to cause disaster are ipso facto rare.
But still, they both seem like growing problems, and not easily solved, given the necessarily increasing distance between the frontier of science and technology and the technical competence the average citizen/consumer possesses.
>and evil combinations of flawed hardware and buggy software that are just right to cause disaster are ipso facto rare.
Not the first time it's happened; Therac-25 shot people with lethal radiation beams because of exactly this. Also, cars with auto-engaging out-of-lane prevention (which also prevents driving straight when there is reflective water or a not-in-program arrangement of lines on the road; "possessed by demons" indeed) - although I think that one happened after/around-the-same-time-as the MAX catastrophe.
Yes, and I expect it will happen more in the future. For a long time we kept software out of life-and-death situations, because it was so self-evidently risky. But we are much more accustomed to relying on software these days, and I think the pressure is rising to allow it into more serious situations, because of the immense savings in cost. I'm not sure we have the right set of attitudes and practices for that not to have some significant tragic outcomes, at least until we develop better attitudes and practices.
Ugh, yes. I recently had a Tesla Model S try to stuff me into the side of a tractor trailer on the Autobahn in a construction zone because the car got confused about which set of pavement markings was important. And then I had the passenger turn that shit *off* right fucking now.
When I bought my last car I was musing over some of these add-ons, automatic braking, lane-keeping and stuff, and the salesman (no spring chicken himself) looked at me and said "Dude...seriously, you've been driving 45 years, are you really going to trust the car to do your braking or steering for you?" and I realized he was 100% right. Maybe a person can develop new instincts of when to trust the car and when not to, but I'm too old for that crap. I don't even use the back-up camera much, I just turn and look like I always have, although I'm glad it exists to show me whether there's a cat or kid's tricycle right up against the bumper.
I didn't read that article, but isn't the main problem that the software lacked the redundancy it needed for safety (relying on one sensor instead of the usual standard of two sensors)? Combine that with the fact that pilots were unable to turn the system off (if they even knew it existed) and it was a recipe for disaster.
Depends who you ask. If you ask me, the main problem is the airframe was flawed, did dangerous stuff easily, and that they tried to fix it in software *at all*. Other people, who don't mind putting their lives in the hands of a computer program, will agree with you, and say the problem is that the program was not well designed, or integrated with hardware.
As I understand it, there's already a lot of safety-critical software controlling car behavior (remember all those Toyota unintended acceleration events?). I agree that in general that this is concerning because it's harder to achieve reliability in software than hardware. In this particular case, though, it's a huge error to force the nose down based on a single sensor reading, and this would still be true if it were a hardware circuit forcing the nose down. It's *so* obviously a bad idea that I've been hoping to hear a report explaining how the hell this could have happened without multiple key Boeing employees being grossly incompetent. Even without the regulator doing its job, a flaw this obvious shouldn't slip through.
The airframe is stable without MCAS. The system was designed to make the handling more 737NG-like, not to correct any particularly dangerous “flaw”.
The pilots were perfectly able to turn the system off. The penultimate Indonesian 737 MAX crew recognized that the automation was about to kill them, in a very slight variation of a common way for automation to kill people in airplanes, so they hit the kill switch that disabled that whole class of automation and flew the plane safely to its destination using the manual backup system. Which is why you never heard of them, or of countless other flight crews doing the same on many other airplanes most of which were not 737 MAXes.
The next Indonesian 737 MAX crew appear to have been clueless buffoons who should never have been allowed to fly an airliner, who in spite of the note from the last crew saying "PS automation is really flaky, don't trust it until it's checked out", never tried to switch to the manual backup.
The Ethiopian 737 MAX crew, appear to have properly recognized the problem, hit the kill switch, and gone over to manual backup controls. Then tried to selectively reactivate part of the automation, and been screwed over because Boeing changed the way the kill switch worked without documenting it. *That* was the unforgivable sin, IMO.
If the idea is that the automation must never ever ever ever fail, get real. Nobody has a clue how to make airliners cheap enough for non-rich you to afford a ticket, without some degree of automation. And nobody has a clue how to make airliners safe enough for you to tolerate them, without human pilots ready to lobotomize the robots and take control. So you make airplanes with lots of automation, because it's much more efficient when it works, and give the human pilots the tools and training and documentation they need to take over when the automation fails
I mean, the MCAS isn't exactly a normal autopilot. Its entire intent was to prevent pilots from inputting dangerous commands i.e. make an airframe safe that is *dangerous to manually pilot* (at least, without specific training, which Boeing wanted to avoid).
Using automation as a failsafe for an untrusted human pilot means you *can't* rely on the human pilot being a failsafe for the untrusted automation - as such, it should be held to a much higher standard than otherwise.
I believe you, but could you clarify something for me: if the pilot has to be there already to take over for the automation failing, why is it cheaper to automate? Can pilots log more hours when they let the plane fly itself as opposed to going manual the whole way? Or are we able to employ a less talented/educated (cheaper) pilot who only has to be capable in a small handful of "automation failed" scenarios and not in the wide away of "operate a plane normally" scenarios?
In the Indonesian case, wasn’t the aircraft also having numerous faults related to airspeed indicators starting on the takeoff roll (before MCAS kicked in)? Also, I thought the report indicated that the captain didn’t think to turn off the auto-trim, but did at least continually flick the trim switch in the yoke, which would override MCAS for a few seconds at a time. This kept them in the air for several minutes, until he handed control to the copilot - without telling him what he’d been doing to keep the aircraft controlled!
The reason Boeing was incentivized to cut corners with MCAS in the first place is that it would have been tremendously expensive to certify and deploy the MAX if they couldn’t pretend it was just a minor update to the existing 737.
The reason it would have been onerously expensive is because the FAA gives the FDA a real run for its money in making innovation insanely slow and costly.
I'm curious - what corners should the FAA be cutting that they are not presently?
FAA certification of a new aircraft type takes years and hundreds of millions of dollars. Pilots getting a “type rating” in a new aircraft costs the airline low 5 figures each initially, plus must be maintained through ongoing training and currency requirements for each type they fly. Having a mixed type fleet is costly - Southwest flies only 737s precisely for this reason. Type rating don’t really distinguish between “basically the same but for one or two small differences” and “completely different airframes”. A MAX under a new type would be just as different from the 737 NG as an ERJ or an A380.
So Boeing has turned the 737 into basically a ship of Theseus at this point, very little is unchanged since the original 60s design, but Boeing is heavily incentivized to make sure it can stay within the original type certificate, since otherwise they can’t be cost competitive. Airbus does exactly the same thing with the A320.
MCAS is probably not something Boeing would have implemented if the MAX was a clean sheet design (the airplane is basically fine without it, but it doesn’t fly just like an older 737). But the FAA’s onerous certification requirements pushed Boeing into design decisions that were made not for best engineering design, but to maintain apparent commonality.
It seems that the FAA mostly went along with this wink-nudge-its-just-a-737 thing because they fundamentally didn’t believe that treating it as if it was a brand new airplane either, but admitting reality would have forced them to do so, because the only thing that takes more time and money than certifying a new plane is changing any significant FAA rule, no matter how out of sync it is from modern reality.
This is an organization that took >20 years to approve a leaded gasoline replacement for general aviation (an industry where high FAA certification costs mean that basically everyone is flying around with 1950s engine technology)
I think the criticism raised by Charlie (and Kevin Drum, and others) just relies on... an unnatural reading of things? If you think a particular government agency should not exist -- if you think that the function it is supposed to perform is one that should not be performed, and therefore no agency should exist to perform it -- that would to my mind normally be described as you being "against" that government agency, and that is how I would normally someone interpret saying they are "against" a particular agency or that that agency is bad. I don't know why one would read it instead as a comment on the quality of the particular staff? I would normally think of government agencies primarily in terms of their function, not the particular people who happen to be executing that function...
I think Scott would disagree that the FDA shouldn't exist. But some details of the rules it must enforce are problematic.
Sure, but my comment generalizes to the case of "the function it is performing is the wrong function and therefore it should perform a different one". You don't have to intend to abolish the FDA to talk in terms of functions rather than the people tasked with those functions.
> But that requires a sort of covert acknowledgment that these drugs are probably being delayed too long - otherwise, these politicians are heroes for taking potentially-dangerous substances and testing them out for the rest of us.
I think there's an internally consistent position where the observer doesn't acknowledge that the drugs are being delayed too long, and just objects to the actor breaking the rules. Basically, in this framework whether the treatment works is moot, the objection is just about fairness and/or the morality of rule-following.
(Not supporting this position, in particular; just providing an observation for why most people don't think politicians are heroes when they cut the line, and also the criticism Biden is trying to pre-empt with his statement that he has to wait for FDA approval before taking his booster.)(
I can think of lots of other things we should use politicians for testing with, though. Perhaps as crash test dummies?
Andrew Gelman does a thing where he writes his content and puts them in a queue to post 6 months-ish later. Maybe that's an option foryour less time sensitive pieces.
> "I probably shouldn’t choose an acronym that nobody will understand until I explain it to them"
Is there any other kind?
Sometimes you can understand an acronym even without having it explained (well, at least I can). For example, if you already know "IIRC" ("if I recall correctly"), you can probably understand the sentence "IIUC, photosynthesis converts light energy into chemical energy".
Point. (Though I'm not sure I would have gotten that particular example if you hadn't primed me first.)
But that particular strategy only seems like it would work for stock phrases your audience already expects you to say based on context. If you are trying to promote an idea that is not already widely-talked about, I'm not sure it can help you.
it had take me minute to understand the acronym, and this after the priming, in the wild, i would have no chance at all.
I think he meant 'explain' as in 'explain the joke and how it is supposed to relate to the situation at hand', not just 'say what the acronym stands for.'
The final example about bupropion-dextromethorphan highlights how misguided it is to solely fund clinical trials through the promise of patent-protection. Presumably the government could make a guess at how likely this would be to work, how valuable it would be if it did, and then the government could have people bid on the contract to run a clinical trial.
Does this happen? Does the government, or anyone, fund clinical trials with the condition that anyone can make the drug if it is approved? The Ketamine example fits here too it seems.
Funding agencies and hospitals will occasionally run trials with generic drugs or to figure out specific clinical questions with already approved drugs-- how long should antibiotic X be given? 10 vs 8 days. Etc.
I don't think this happens nearly enough, however.
> a YIMBY analogy by dubbing my position YIMCS (“Yes In My Circulatory System”)
Call it YIMBI: Yes In My Body Innards!
Yes In My BodY! Ok, perhaps overloading the acronym isn't the best plan either.
I'm kind of baffled how you can invoke the "normal knowledge that normal people use" as some kind of accessible source of information when nobody has the first clue how to gather or assess that. Normally among pundits it operationally reduces to "what me and the people I know/read/like think" but that leads straight to the Pauline Kael effect, and some of the unpleasant divisions between "expert opinion" and what vast swathes of regular folks think that we see today on any number of issues.
If nothing else, consider that the "knowledge of normal people" about which we have cared deeply for several centuries is "who should run the country?" and for this purpose we have this vast apparatus of elections, at various levels, terms of various sizes, representation or not, not to mention polling and focus groups and prognostication -- and yet we *still* are routinely surprised by the outcomes, and debate furiously just how genuinely representative they are.
What hope is there of actually discovering what the "knowledge of normal people" actually is on the question of whether Drug X should/should not have been approved, or whether the FDA should be "stricter" or "looser" in general, for various definitions of "strict" or "loose?"
I feel like all you're really saying that's 100% defensible here is that given time, a consensus will emerge, and using the clarity of hindsight we will all be able to say decision Z or T of the FDA was right/wrong or hasty/slow. But noting that hindsight is easier and more accurate than foresight is rather a truism and (barring the invention of time machines) gives no clue on how to improve the situation going forward.
Scott's position is that the consensus often emerges before the FDA makes a decision.
This is not a mechanism that Scott is proposing, it is a standard. If everyone (in the medical community?) already knows that e.g. covid vaccines will be approved for children and the FDA hasn't allowed it yet, then the FDA is moving too slow.
The FDA should be no slower than hindsight.
Yes, well, the gist of what I said is that I have very strong a priori doubts that this is true -- that one can empirically measure a reliable "consensus" (among whom? would be the first question) before the FDA makes a decision.
As a rule, people react to events, they do not work proactively to form an opinion on events that have not yet happened yet. Exempli gratia, I doubt except among those most intimately concerned (patients, physicians, researchers) anyone even tried to form an opinion on whether aducanumab should be approved in advance of the event.
And if the "consensus" which we intend to consult is merely among the small community who are intimately engaged ahead of time, because of assorted intersts, but...er...not the *same* small community that is intimately engaged in FDA approval, this (1) does not meet the definition of some broad-based "general commen-sense person" community, and (2) isn't obviously better than what we already have.
Perhaps I should add that I don't doubt a consensus may emerge in some random community or other -- e.g. "all the people I know" -- but as I said at the top, that is a very distinct thing from a general broad-based consensus, and the former should not be mistaken for the latter.
I think the general situation can be 'incentive hacked' from above in a way that simply pressures the experts in the correct direction
Give everyone at the FDA a 10% raise and then make 20% of their paycheck dependent on their performance in terms of a chosen cost to benefit or cautionary principle standard. Something like "10 people's lives are not saved due to overcaution so that 1 person's life is not lost due to lack of caution"
If the FDA is responding to huge overcaution incentives where nobody notices if they don't approve something and good things do not happen and someone loses their job if they do approve something that goes wrong then the large paycheck incentive may fix that and at least might serve as a red flag that the process has problems
At the same time if the FDA is not internally able to approach the quantified goal they themselves will be the ones waving the red flag and identifying what the obstacles are
Who manages the managers? Who watches the watchmen? This just declares an intent to fix things and says “someone’s job will be to make the correct decisions, somehow? And they’ll be responsible for the actual decisions”. It doesn’t fix either the “what is the actual principles we can teach and carry for what to approve and how and when”, and “how does your incentive system not bend when Eric topol says to not rush the vaccine or your expanded approval system (whether or not it really did, doesn’t matter, just the pressure) kills ten black babies in underprivileged communities.”
It would be straightforward enough to be able to assess whether the performance measurement of the FDA was reasonable. We all know that pausing J&J may have saved a few lives and definitely cost, what, thousands? That one at least is pretty bright and easy
One thing the incentive pressure would likely immediately do is cause the FDA to choose a number of other regulatory agencies like the EU's and provisionally or fully approve everything they do because the likelihood is they are overcautious as well. Suddenly the regulatory problem is reduced to whichever agency is the best, assuming the primary problem is in fact overcaution. (I take this almost for granted given the incentives that are at play but I understand that emotionally there is a constituency for allowing 100 or 1000 lives not to be saved so one does not die from undercaution because for some it does feel more wrong for someone to die as opposed to 100 not being saved. What the FDA might do under pressure from a "ok, but here's how far you can go with that" incentive is open up provisional approval that allows people to use drugs that are not fully approved under their own volition and request that the calculation of their performance on not approving that drug yet be suspended since people have at least been given the choice to take their own risk)
Re: your second response below I'd say 20% of someone's paycheck is enough to become the primary incentive in their work. If it doesn't we should hear about just what the problem is because there will at least be an incentive to complain
> It would be straightforward enough to be able to assess whether the performance measurement of the FDA was reasonable
Not really? That sounds somewhat hard. Let’s say the FDA approved all ten Alzheimer’s antibodies right now, but didn’t mandate insurance prescription. And then let’s say they opened a rolling trial option where up to 10k total patients per drug may pay up to $100 each to participate in a trial for drugs that have passed animal safety checks and human safety model checks, to speed up development and use that data for phase three trials, and then this is used for more monoclonal antibodies and also for early likely to fail 3D printed organ test transplants in terminal patients who will die anyway to speed up testing. Should that be rated as Good or Bad by your agency? Is it obvious? Not really
I don’t like the idea of companies regulatory shopping for which of seven agencies with partial mutual recognition of approval is the most generous or corruptible. In some areas that’s fine, but for this one eh. I’d be happy with re using trials and evidence used for other countries if they pass our standards but I’m pretty sure we already do that
20%+ paychecks are not enough to be a primary incentive if, for instance, you will have your agency power taken away if you do the thing, by public backlash. Might also be thinking more of a corporation and less about the actual structures at play in the FDA - who’s making the decisions? Who’s salary so you increase? The political appointed head? Political appointees of all people will have other incentives, and potentially limited ability to fight the careerists if they even want to.
Also organizational persistence and power and public perception and laws and industry relations probably >>>>> paychecks
I think 'voting is wonky so we can't learn public opinion about anything' is a bad take.
First of all, in our two-party system, the parties autoamatically re-align themselves so that each side is always capturing about 50% of the population, making the decisions close and hard to call all the time, and there are massive incentives on all sides both to pervert the process of finding out what people want and to misinform people about what they should want. This is probably the hardest version of this problem imagineable, because of the unique circumstances surrounding it.
If you wanted to find out if, say, the majority of people believe that rain is cause by pixies crying when they see a clown get hurt, it would not be at all difficult to get this information. You would do a phone survey with a few hundred or thousand respondents, the outcome wouldn't even be close, and the percentages you get would be very close to accurate.
Those are the two extremes of the easiest possible problem vs the hardest possible problem, but I think that most things you might care to ask about are closer to the crying pixie end than to the voting for president end of the spectrum.
I think it *feels like* the population is really divided on most issues and it's hard to ind out what they really think, precisely because we spend all of our time talking about controversial issues that are tied up with politics and active manipulation. But the vast majority of issues you might ask someone about are not controversial in this way, and on those I think it's not that hard to get a public consensus.
I certainly agree on many issues people *can* be assessed for their understanding and viewpoint pretty easily. But not in the slightest with respect to medicine. People feel *way* more strongly and emotionally about personal health than they do even about who is President. You can take the most rabid Trump-really-won fanatic and the die-hard AOC fanboi and all their political differences will vanish like a pricked balloon if the question before them is how to prevent themselves, a spouse, or a child from snuffing it from a brain tumor, and whether Drug X will do that or not, and whether it should be available, cost what it does, et cetera. Health care is *even more* screwed up with passions and illogic than politics.
I find a rather different interpretation of Nestor's shuffle about the FDA on (the ever inerrant) Wikipedia: https://en.wikipedia.org/wiki/John_Nestor ::
""On March 19, 1972, the FDA reassigned Nestor to its Office of Compliance, as part of a reorganization in which Civil Service doctors were replaced as drug approvers by consultants who were often affiliated with commercial companies. Nestor began a grievance proceeding. A review panel determined that senior FDA officials gave "misleading" testimony against Nestor and concluded that Nestor was due an apology and appropriate duties.""
By analogy, consider criminal trials.
The final decision is made by a bunch of men-on-the-street: a jury. We decided that, even though justice system officials know a lot more about crime, it is too difficult to get rid of their inherent biases, so a group of men-on-the-street does better.
If we take this analogy even further, we could have juries for the FDA. The experts can make their arguments, but the decision would ultimately be decided by a group of men-on-the-street. Random people would get called to make decisions that effect the life and death of their fellow citizens.
While I think that this is interesting, I'm not convinced that it is a good idea. The FDA is often too slow to approve good drugs and the jury system is not exactly known for its speed.
Judges make plenty of decisions in criminal cases, and juries are notorious for not understanding and making somewhat mistaken decisions in trials involving detailed and complex technical evidence or issues. (Judges also do that, but the FDA’s experts are subject matter experts)
I don't think the analogy is especially applicable. The role of a jury is a trier-of-fact. That is, their job is to assess the believability of the witnesses both sides put on, and decide who to believe. The *judge* is the one who decides what the law says, i.e. given this pattern of facts on which the jury agrees, what is the crime (if any) that was committed, and what is the appropriate result?
There's a very decent reason to believe that a panel of average strangers are well positioned to collectively decide whether Witness X is telling the truth or lying. It's a generally-distributed skill, for which there is no known education or technology that could do a better job, it's a collective decision, so individual flakiness is averaged out, and the people involved are very detached from the rest of the judicial process, so they are not influenced by considerations of the sentence, of the effect on other defendants or cases, of the nature of the law, et cetera -- all of which can have powerful effects on lawyers, prosecutors, and judges, the experts in the bowels of the system.
There isn't much that's analogous in the FDA's decisions. A major part of what a criminal trial is about is deciding who's lying, and there is (one hopes!) essentially none of that in the FDA's decision process.
Furthermore, the assumption that would seem to underly this, that an *individual* man on the street has a better judgment of the details than an individual FDA apparatchik, seems dubious a priori and anyway isn't part of Scott's proposal -- I believe he's arguing that some vast consensus, the wisdom of the crowds, so to speak, can exceed the good sense of a much smaller number of FDA experts and bureaucrats.
That is, the key change is to make the number of people who influence the decision much, much bigger. (This may also be the assumption underlying the idea of taking guidance from approvals in other countries -- basically, if the EU and UK and Japan all approve a drug faster than the FDA, maybe the wisdom of the crowds is telling us something.)
Honestly, I think juries are in the wrong place. I'd rather put the "committee of men-on-the-street" as one of the houses of parliament (Congress in the US), where they can tear up criminal overreach at the source.
It sounds to me like you're describing Sortition: https://en.wikipedia.org/wiki/Sortition. Examples of Soritition in real life: Taiwan's vTaiwan system (https://www.technologyreview.com/2018/08/21/240284/the-simple-but-ingenious-system-taiwan-uses-to-crowdsource-its-laws/) and Ireland's Citizen's Assembly (https://www.electoral-reform.org.uk/the-irish-abortion-referendum-how-a-citizens-assembly-helped-to-break-years-of-political-deadlock/).