Middlebrow science
When I studied economics at Chicago in the 1970s, I was taught that the FDA was a highly flawed institution, which was structured in such a way as to avoid errors of commission at all costs and not worry so much about errors of omission, even if 100 times more consequential. That’s the system, and we’ve known it for 50 years. The current vaccine fiasco was entirely predictable. This isn’t “Monday morning quarterbacking”.
I presume that FDA officials are decent people, but they are embedded in a bureaucracy that operates with a really bad set of incentives. The solution is not to replace one FDA bureaucrat with another, it’s to take government out the the process of deciding what vaccines are acceptable, who gets them first, and at what price.
Alex Tabarrok recently posted some excellent comments on the vaccine fiasco. Here’s Sure:
The likelihood ratios for RCTs and approval mechanisms are powerful. But we are talking thousands of deaths per day. The odds that these tests will remotely alter management decisions is nil. It is malpractice to delay life saving treatment on tests exceedingly unlikely to change management decisions.
And remember the UK is not seeing horrid outcomes for doing this for a while now. A lot of theoretical failure mechanisms are now off the table.
Science is wholly about building a reliable model that accurately predicts future outcomes of current actions. While doing the actual experiment is the gold standard for knowledge acquisition, it is not the only option and in cases like this pandemic is not sufficiently better than past data to merit waiting.
As far as the regulators. I work with some of them directly. They are not overburdened to anywhere near the degree that the frontline clinicians have been hit. When I ask them to explain their cost benefit calculations, they have none. Not I cannot follow them. Not I disagree with them. They have done not an iota of math to justify their course of action.
Sorry, but I believe in evidence based medicine, not eminence based medicine. If you as a regulator cannot explain to me in technical terms the math behind your decision process, even if only back of the envelope, you are not worth putting in charge.
Approve all the vaccines, FDF, fractional dosing trials, and first dose followed by variolation trials should all be done now.
Here’s Tom Meadowcroft:
Scientific researchers search for the truth. Medical clinicians use limited data balance cost and benefits in the face of uncertainty to save the most lives.
When searching for the truth, it is important to have high standards of statistical significance, integrity, and patience, because credibility and a reputation for integrity is everything. Every academic knows that a retracted paper or an accusation of playing fast and loose with statistics can be the death knell for a career. As a result it is prudent to be very certain before publishing. Public health officials, particularly those in charge of approving vaccines, dread the possibility that a vaccine that will be given to millions of healthy people, often children, to prevent diseases where death is rare, which could harbor some flaw that causes a hundred avoidable deaths; they seek the highest standards of proof of safety and efficacy before approving such a vaccine.
But a pandemic is not a search for truth, and a COVID vaccine administered in the midst of a pandemic is very different than a measles vaccine administered to 2-year-olds. The pandemic makes these decisions for FDF or for vaccine approvals into clinical decisions, where health professionals should be balancing the certain benefit of reducing the thousands of daily deaths against the uncertain cost of the possibilities of harmful side-effects and uncertain details of efficacy (when does immunity kick in, how long does it last, how valuable is a booster) that additional months of testing and trials would reveal more clearly.
Public health researchers, academics for the most part, lack the ability (and courage) to make the sort of cost/benefit analysis with necessarily limited data that clinical physicians make every day in examination rooms. Any good clinician, faced with the citizenry of a country as their patient, would have opted for FDF, the AZ vaccine, and quite likely reduced doses by the start of the year. Because we are stuck with academics and administrators as our decision makes, unable to see beyond their usual routine of searching for the truth and protecting their reputations, thousands more will die.
PS. For those of you who think we are doing fine and that Israel is an outlier, you need to add the Navajo Nation to the list of outliers:
The COVID vaccine rollout in the Navajo Nation has been highly successful, already surpassing its original goal to have administered 100,000 shots by the end of February. It’s an impressive number, given that there are an estimated 175,000 people living in the Navajo Nation. . .
As of Feb. 11, the IHS says, a total of 113,345 COVID vaccinations had been administered in the Navajo area.
Tags:
23. February 2021 at 11:57
I wonder if it is pure “coincidence” that such small populations as the Navajo or Israeli Jews are so far ahead in vaccination.
The most likely reason is really the relatively small size of the population. There are probably many populations of this kind, so you will always find positive outliers.
But maybe, just maybe, it also has something to do with being more sensitive to what is essential in life. Perhaps they are more down to earth and more rooted in the real world and its dangers.
23. February 2021 at 13:10
Its also easier to effectively implement public policy in homogenous populations. Messenging is easier since theres a lot more shared culture and generally homogenous populations have higher trust.
23. February 2021 at 16:38
I’ve found myself gravitating towards the information coming from virologists, aerosol scientists, clinicians and economists instead of epidemiologists during the pandemic. Their models were either richer or more flexible or both. I, of course, am probably selling epidemiology short, but it seems the discipline needs to get out of the middle ground they occupy between the deep domain knowledge of the virologists, aerosol scientists and clinicians and the comfort with evaluating complex systems possessed by the economists.
23. February 2021 at 17:33
Scott, Christian,
I’m sympathetic to the idea of fast and creative vaccination strategies, the idea that public health officials are slow and bureaucratic etc. That said, as a general principle, it is not automatically OK throw established standards overboard in an emergency. Same reason why knee-jerk lawmaking designed around freak accidents or salient crimes is a bad idea. Yes it is possible, in-principle, that the novel vaccines generate auto-immune disease later on, and this has been documented for RNA vaccine approaches long ago. Is it likely that there is a mechanism where this would hit a large fraction of the vaccinated with a long delay? I would hope not so and I’d guess it’s unlikely. But the trade off of future risk vs immediate benefit is not as slam-dunk as it can appear right in the middle of a crisis.
But there’s something else that gives me pause.
Have you noticed that China had the first vaccine, Russia was also quick. Russia had fast track approval for its own vaccine. China apparently did not approve its own vaccine until much later. Neither started quick and comprehensive vaccination campaigns any earlier than the West. And at least China surely has the “State capacity” to get things done top-down if its government wants it to happen. Or consider Singapore: small, wealthy, extremely organized. Only front line workers have been vaccinated starting January. Seniors are starting to be vaccinated just now, end of February. Overall per-capita vaccination rates as of now are about the range you have in the EU, i.e. very modest. Why do you think a number of such countries with deep resources and strong state capacity are not in a particular rush to vaccinate? If I had to guess, I’d say the medical and scientific community that advises decision makers is cautious pretty much everywhere, even where there is no FDA, or messy federalism or what have you in the West.
23. February 2021 at 22:46
Christian, Most American states have fewer people than Israel, and vaccination in the US is handled at the state level.
Sean, Maybe it shouldn’t be a public policy.
Carl, Good point.
mbka, What I find most dispiriting is that the public health people seem unable to even explain what they are doing. If they had an explanation and I didn’t agree with it that would be one thing, but they don’t even seem to have any sort of cost benefit analysis for me to disagree with. What’s the model?
24. February 2021 at 02:56
Abolishing the FDA would be a big step, but how about requiring them to publish their cost benefit analysis?
24. February 2021 at 06:24
Well, Biden did blame Trump for something. However, about 90% of the vaccines that have been distributed to the states have been administered. So the logistics within states seem decent (says “Beckers Hospital Review”). The problem is only about 10-20% of what is needed (2 shots makes it 10–but 1 shot is still beneficial) has been shipped. This seems absurd.
Only 60% of the doses Moderna has given the Fed’s, have been administered—-Pfizer and Moderna say by March there will 220 million more doses delivered by March—-(that is next week) to the Fed’s. How fast can Fed’s ship? Don’t know—but they supposedly at least can track what is shipped. It does not seem fast as Moderna’s full Fed Ship,Mets have not gone to states.
J&J vaccine is as effective in preventing hospitalizations and deaths as the other 2. It is viewed as safe. How long before it is finally approved and shipped? As of yesterday Fed’s have not said. But let’s get the 220 shipped and administered.
Not a logistics person. But if the Government cannot do this efficiently what is their value?—-each day of delay kills people. I have no reason to blame Biden—-because how would I know? I do feel confident, as Biden has shown already, we would be blaming Trump. Of course it does not matter. I would like to hear some summary of a plan of delivery.
See, I am like Scott—cannot let go,of Trump!
24. February 2021 at 07:59
@Michael R:
JNJ vaccine just approved:
https://www.cnbc.com/2021/02/24/covid-vaccine-fda-staff-endorses-jjs-single-shot-for-emergency-use.html
24. February 2021 at 08:25
If we didn’t have the FDA as a gatekeeper in a drug approval system, what would we have its place? What nation could be a good precedent?
At the very least it would be good to see the responsibilities for all Food matters regulated by one dedicated government agency and all Drug and Medical procedures regulated by another dedicated agency.
Why do we still have a Bureau of Alcohol, Tobacco, and Firearms? Or a DEA? Ooops, I’m going down a path that leads to me tearing out more of my hair.
24. February 2021 at 08:38
As one who spent his working career in the biopharmaceutical industry I find that most of those speaking out against the FDA have no clue at all how the Agency works. Nor do they have any comprehension about vaccine development and manufacturing. That we have two mRNA vaccines that went from conception to broad scale deployment in less than 10 months is truly astounding. Not only did they need to figure out how to make these at scale (not trivial despite the statements of such economists as Professor Cochrane who on a recent podcast said we had a vaccine in January of 2020) but they also had to show they worked and that there was a margin of safety. It is the safety aspect of this that required the most time as the number of patients is large relative to pharmaceutical trials.
FDA did issue guidance on Monday that future COVID-19 vaccines that address variants will not need a full scale clinical development program but rather small studies on several hundred patients that demonstrate necessary immunogenicity against the variant as long as the manufacturing process is unchanged. This is exactly what is done with the seasonal influenza vaccine and it should be possible for Pfizer/BionTech and Moderna to create and make available new variant targeted vaccines in a matter of weeks rather than months. What is wrong with this approach?
24. February 2021 at 08:42
@msgkings – The J&J vaccine has not be approved, only the FDA review has been announced. Authorization is expected shortly.
24. February 2021 at 09:10
@Alan G:
Right, that’s more correct but I was just responding to his comment on the J&J one, it’s getting approved in a matter of days.
And I agree with your prior post, I don’t see why there’s such hatred for the FDA. Some hard libertarian positions go too far. A big part of public health is getting the public to trust the measures needed. As bad as it is now (anti maskers, really?) it would be far worse without some authority in charge dedicated to making sure things are safe and effective, even if that creates delays.
24. February 2021 at 10:39
It’s déjà vu. I.e., 2018 all over again. Listening to the stagflationists lie, those that advocate N-gDp Level targeting or FAIT, Fexible Average Inflation Targeting, or 1970’s stagflation reincarnated.
Remember: “Rethink 2%”
http://bit.ly/2s67De9
24. February 2021 at 10:58
Also agree with much of what AG said. If there was no FDA, the magic oil people would be running around claiming dabbing some lavender oil on your nipples prevents covid.
But the borg hasn’t accepted that pandemics change the calculus (if they are even doing any).
24. February 2021 at 11:10
I can get behind small tweaks and reforms to the FDA, but I am not in favor of eliminating it or overhauling it. I feel that the pharmaceuticals need even more oversight than there is today, because the government-granted patent monopolies that drugs receive encourage falsifying data and other deceptions. (I don’t understand why free market proponents never address government-granted patent monopolies) As a simple example, look up the list of controversies on the Wikipedia page of any given pharmaceutical company.
The FDA needs to be stringent about approving any given mass-consumed drug, even given our current pandemic, because if there is any significant long-term side effect or significant number of deaths that result from it, it would set back public trust in vaccinations (for all diseases) for decades to come.
And if someone wants to ask me “so you’re fine with hundreds of thousands of additional COVID deaths as a result of a slower vaccine rollout?”, I reject that attack. There are a multitude of different actions and policies we could have taken to better manage the virus until vaccines became available.
24. February 2021 at 12:09
Scott is 100% correct here. Most commentators, even on this blog, are pretty clueless. I have nothing against the FDA, let them do their job, let them test and evaluate as many drugs as they want to.
Ultimately there is one big problem with the FDA: the FDA can prohibit people from buying and taking a drug. By what right should they be allowed to do that? Why shouldn’t people be allowed to decide for themselves whether they want to buy and take a drug or not?
@Ryan
This is constantly criticized, what straw man world do you live in? Are patents not a massive (and wrong) regulation? And because this regulation is bad, you want more of the bad, in the hope that bad plus bad equals good?
There should only be clear rules and laws that stop anti-competitive behavior. There seems to be not enough competition between pharmaceutical companies in the US. They pretty much cornered the market.
In the US, for example, it is legal for pharmaceutical companies to pay competitors not to bring low-cost generics to the market. An absurd anti-competitive practice. I have never understood why this is legal in the US. This is pretty much one huge cartel, illegal monopolization, price fixing, corruption.
24. February 2021 at 12:32
@Christian List – IIRC you practice medicine in Germany. Does Germany allow patients to buy and take drugs they think they need? You are also mistaken that FDA prevents people from taking drugs they need, the FDA only approves drugs. It is up to the prescribing physician to make the decision whether the patient needs or should get a particular drug. In addition, the US does not have price controls that virtually all Euro countries have. Finally, the US has many more drugs available OTC, not requiring a prescription, than almost any other country.
You all misunderstand the generic drug issue you cite in the final paragraph. The loophole you refer to only slows down the switch by six months and the way the law was written the loophole was not anticipated. It’s more complicated to explain in a blog response but Google can be your friend if you really want to understand the issue.
24. February 2021 at 13:35
I’m curious (actually curious and not rhetorically curious) how those advocating the elimination of the FDA envision safety and efficacy testing will be financed. Do you foresee:
– an Underwriters Laboratories equivalent financed by the health insurers
– pharmaceutical industry backed boards that certify safety and efficacy
– something else
24. February 2021 at 13:55
Everyone, For those saying we should “trust the process”, which process? The countries with 3 per 100 vaccination? 10 per hundred? 25 per hundred? 85 per hundred? All of them?
Alan, I know how the process works, and I know it can and should go faster in an emergency.
Ryan, You said:
“I don’t understand why free market proponents never address government-granted patent monopolies”
They do, read Alex Tabarrok.
You said:
“The FDA needs to be stringent about approving any given mass-consumed drug, even given our current pandemic, because if there is any significant long-term side effect”
The FDA never tested this question for Covid vaccines. Hence the “emergency” approval.
Carl, When Milton Friedman was asked what a free market school system would look like, he said something like, “If I knew the answer I won’t favor private schools, I’d just instruct public schools to teach that way.”
But yes, probably something like UL, paid for by the drug companies.
24. February 2021 at 19:28
Great quote by Friedman.
25. February 2021 at 09:14
The FDA claims to be weighing costs and benefits: “For an EUA to be issued for a vaccine, for which there is adequate manufacturing information to ensure quality and consistency, FDA must determine that the known and potential benefits outweigh the known and potential risks of the vaccine.” https://www.fda.gov/vaccines-blood-biologics/vaccines/emergency-use-authorization-vaccines-explained
25. February 2021 at 10:20
foosion, C/B analysis also should apply to the speed at which decisions are made. Why so slow to approve J&J?
26. February 2021 at 07:29
@Alan
You have a grotesque way of phrasing things. Every new drug has to be approved by the FDA since 1938. This process is quite expensive, bureaucratic and time-consuming. What the FDA has not approved cannot be prescribed, for example because of insurance and liability reasons, but also because the drug is usually not on the legal market. So what are you even talking about here. It sounds completely grotesque in my ear.
Scott Alexander has 2-3 articles with examples of what tricks the US bureaucracy and pharmaceutical industry use to unknowingly or deliberately prevent the sale of cheaper drugs.
Maybe you read the articles and then we talk again.
Actually, with this huge market, the prices of drugs should be among the cheapest in the world. So why are they so expensive. What is your explanation?
26. February 2021 at 07:49
Remember when snake oil salesman were selling unlabeled concoctions with morphine in it to babies for teething? There is a roll for a dedicated body to watch out for this and enforce labeling laws and such. But, like with the fed at times by gone, they are screwing things up right now. Perhaps for different reasons but come on… why are these reviews taking so long? And Scott and Alex and all them are right… on what mathematical basis are you basing your actions on? What is going on here? Pick up the pace ace! And first doses first too. Wtf?
26. February 2021 at 08:07
Random unrelated comment:
It’s almost March. 500,000+ covid deaths. Vaccinations being slowed down by failure to follow the math. New variants maybe more contagious are starting to take over. It remains to be seen how effective the vaccines will be… tho boosters are being developed.
Youyang Gu’s models suggest Ro is currently 1.0 and holding steady at that to maybe declining. This implies we are at the inflection point of the logistic. Thus total deaths should be 1M all said and done (given a logistic).
Pace must be picked up!
26. February 2021 at 09:14
Students, Yes, the pace of vaccination is a disgrace. It’s actually slowing down.