High Rate of Negative Cancer Trials Raises Concerns

More than 80% of recent phase III trials in cancer conducted over a 10-year period failed to show clinically meaningful improvement in overall survival (OS), according to a review of the data.

Among 187 trials with a primary endpoint of OS, 131 did not achieve statistically significant improvement. Of the 56 studies that met statistical significance, 33 failed to meet the new or experimental therapy’s projected effect size (magnitude of improvement over the control group).

In 216 studies with progression-related survival (PRS) endpoints, 134 (64.4%) achieved statistical significance, but a fourth of the statistically positive studies failed to produce the projected effect size, reported Changyu Shen, PhD, of Harvard Medical School and Beth Israel Deaconess Medical Center in Boston, and colleagues, in the Journal of the National Comprehensive Cancer Network.

“Our study highlights the need to more efficiently identify which new therapies merit phase III testing,” Shen said in a statement. “In order to sustain the rate of innovation in cancer therapeutics and ensure that our patients have access to effective yet affordable therapies, the clinical trial pipeline in oncology must be efficient and accurate. Our work shows that in the past 10 years, this has not been the case.”

“Our study shows that reducing false-positive errors by imposing more stringent statistical threshold in phase III trials is not likely to be practically feasible. A better strategy is to rethink the process that leads to the decision of moving a new therapy to phase III testing to begin with. More research is needed in this regard,” Shen added.

The review evolved from concern about whether phase III clinical trials that demonstrate statistically significant improvement with new cancer therapies represent true improvement in clinical outcomes. One recent study showed that many cancer therapies have received approval on the basis of trial results that hinged on a small number of clinical events. Another study showed that 40-50% of phase III trials in immunotherapy that achieved statistical significance did not produce the projected effect size for OS or PRS.

A study of cancer therapies for solid tumors approved from 2002 to 2014 were associated with OS and PRS improvements of 2.1 and 2.5 months, respectively.

“These studies raise concerns about the number of false-positive errors, which we define in this study as new therapies that demonstrate statistically significant benefit but do not actually achieve the effect size that the trials were powered to detect,” Shen’s group stated.

Conversely, a review of data from phase III trials showed an overall success rate of 48.5% for trials in oncology, as compared with 70% for phase III trials in all other therapeutic areas combined. The disparity could reflect a large number of false-negative results in oncology trials (statistically negative but achieving the projected effect size), the authors noted.

What the Data Showed

To examine the issues surrounding clinical and statistical significance of clinical trials in oncology, Shen and colleagues examined data for phase III trials in oncology from 2008 through 2017. They included only industry-sponsored trials statistically powered for superiority and that had primary endpoints of OS or PRS. They excluded studies involving fewer than 100 patients.

The analysis included a total of 362 trials, 187 that had a primary endpoint of OS and 216 that evaluated a therapy’s impact on PRS. The trials primarily involved patients with lung, breast, gastrointestinal, or hematologic cancers.

Among the trials that had OS endpoints, 56 (30%) met criteria for statistical significance. However, in 33 of the 56 positive trials (58.4%), the results did not achieve the projected effect size (false-positive), leaving just 23 trials that met both statistical and effect-size criteria. One trial that was statistically negative achieved the projected effect size.

“Essentially all experimental therapies from negative trials are correctly identified as not having sufficient efficacy,” the authors stated. “Aggregating all OS endpoints from trials with and without statistically significant improvement (13.1%) of the 187 OS endpoints tested in phase III trials have sufficient efficacy.”

The review of PRS endpoints yielded somewhat more positive results. Of the 139 (64.4%) statistically positive trials, 75% reached the target effect size. The false-negative rate was slightly higher, as 4.2% of trials met the target effective size despite being statistically negative.

The high rate of false-positive phase III studies raises concerns about safety, efficacy, and cost of new cancer drugs, the authors said. Rather than alter the design of phase III trials, they suggested a solution lies in reducing the false-negative rate in phase II trials that determine which therapies advance to phase III testing.

“Tolerating lower power in phase II trials would reduce the number of futile phase III trials,” they continued. “Because phase III trials are much larger phase II trials, this strategy would effectively expose far fewer patients to therapies without sufficient efficacy, which is ethically desirable.”

Trust the Process

The high rate of false-positive studies is concerning and should be viewed as a call to action for clinical researchers, said Crystal F. Denlinger, MD, chief scientific officer of the National Comprehensive Cancer Network.

“We really need to think about ways in which we should prepare better for phase III testing,” she told MedPage Today. “As opposed to rushing or moving quickly into phase III testing, we need to be careful and deliberate in how we get to the phase III clinical trial.”

Experimental therapies that make the jump from positive phase I results to phase III testing are a potential contributor to false-positive results.

“We have a process for clinical trial development from phase I to phase III, and I think the recent trend of jumping from phase I expansion to phase III trial may have allowed some therapies to move forward in a manner that was quicker than maybe they should have,” Denlinger said. “The efficacy that you’re seeing in a phase I trial is really in a very well controlled population.”

“When we go from phase I and a very controlled environment to a phase III study, which is more representative of what we anticipate seeing in the real world, we can lose something in that translation,” she said.