Billlions and Billions of Dollars

I've always suspected, just from acquired memory, that the real reason behind pharma's claim that it cost $X billion to bring a "new drug" to market is that drug companies simply don't pull the plug when the data say the drug has little chance of working and/or getting approved. For those who may not know, in the US, there are three sanctioned levels of trial, not surprisingly called Phase I/II/III. These are trials of the drug in humans. Prior to Phase I there are pre-clinical lab tests to, at least, demonstrate that the drug works chemically, biologically in glass, and biologically in non-humans.

Once FDA is convinced that the compound is non-harmful, or least non-fatal, in non-humans, clinical trials can begin. In general:
Phase I -- safety
Phase II -- safety and dosing, and possibly efficacy, in small trials
Phase III -- efficacy in large trials

As a rule, at least two PIII trials demonstrating statistical efficacy and clinical benefit beyond current therapies are needed to ask FDA for approval. The key points in these trials:
1 -- sponsors (aka, drug companies, mostly) are not required to provide the public, or investors for corporations, all data generated or FDA correspondence during development
2 -- FDA is not allowed to release much, if any, data until such time as it makes a marketing approval decision, and then the non-approval (aka, CRL) may be vague

The result of this is that drug companies often continue pouring money down a rat hole. It's what they do. That's my recovered memory of watching the drug business for the last decade or so. Finding clear data on how many drugs with failed/marginal Phase trials are then sent into the next Phase is difficult. Not the sort of information drug companies want publicized.

Part of the problem may just be a naive` view of stats, in particular what a p-value means. And, no, I don't say that as intro to pumping Bayes in clinical trials. Not even.

Then I found this piece. All is revealed.

And, of course, add to all that the entirely avoidable, but nonetheless remarkably prevalent, tendency to progress agents into Phase 3 that did not actually achieve positive Phase 2 findings (at least without the help of unjustifiable post hoc analyses).

So, here is where all that moolah goes:

If, for example, your primary end-point reaches statistical significance but every secondary end-point suggests no effect, its time to suspect the False Discovery Rate. Put another way, don't let the data from one single experiment (however important) dominate the weight-of-evidence. The attitude "well, the trial was positive so it must work - so lets plough ahead" may well be tempting, but unless the broader picture supports such a move (or the p value was vanishingly small) you are running a high risk of marching on to grander failure.

Leading to his conclusion:

Failures in large, expensive Phase 3 trials are the principle cause of poor capital productivity in pharmaceutical R&D. Some of the reasons for failure are unavoidable (such as, for example, the generalization problem). But the False Discovery Rate is most definitely avoidable -- and avoiding it could half the risk of late-stage trial failures for first-in-class class candidates. That translates into savings of billions of dollars. Not bad for a revised understanding of the meaning of the humble p value.

But, of course, drug companies won't do that, since they get to keep their bloated bureaucracies only if they continue to do trials. Cutting off the losers in PI or PII does nothing to promote that. So, they won't.