What Data? Alice In Clinical Trial Wonderland

What Data? Alice In Clinical Trial Wonderland

January 5th, 2012 // 1:35 pm @ marquee

A remarkable series of studies published this week in BMJ discloses various problems with clinical trials – or more specifically, publication of clinical trial data. As it turns out, there is a lot that goes missing. As the BMJ editorial laments: the articles “confirm the fact that a large proportion of evidence from human trials is unreported, and much of what is reported is done so inadequately.”

Here are some of the findings: One paper examined unpublished evidence of existing meta-analyses of nine drugs approved by the FDA in 2001 and 2002, but found identical estimates of drug efficacy in only three of 41 cases, or 7 percent. Estimates of efficacy were split evenly split in the remaining cases when unpublished FDA trial data was examined (read more here).

Another example: The Food and Drug Administration Amendments Act (FDAAA) requires mandatory reporting of summary clinical trial results within one year of completion on ClinicalTrials.gov. But one BMJ paper found that only 22 percent adhered to the mandate. Generally, later phase trials and industry-funded trials were more likely to report results (look here). Another recent study found even lower compliance (see this).

Still another paper looked at trials that were funded by the National Institutes of Health between 2005 and 2008, and found that only 46 percent were published in a peer reviewed biomedical journal indexed by Medline within 30 months of trial completion. And three years later, one third still remained unpublished (please look here).

Meanwhile, another paper found that at least 3,000 records of randomized controlled trials entered into Medline between 2006 and 2011 were indexed, which means that many trial will be missed when entering the commonly used research term – randomized clinical trial – into the database. As the authors noted, important evidence in searches – design and methods, baseline characteristics, long term follow-up, and secondary data analyses – can be missed (you can read the full paper here).

Finally, yet another paper examined the potential for publication bias, data availability bias, and reviewer selection bias in recently published meta-analyses that use individual participant data. Only nine of 31 meta-analyses included individual participant data from unpublished studies in their primary meta-analysis, and the potential for publication bias was discussed or investigated in just 10. Meanwhile, 16 did not obtain all individual participant data requested, and five did not mention this as a potential limitation. Just six examined how trials without individual participant data might affect the conclusions, and reviewer selection bias was a potential issue in nine (here is the paper).

“There is an ‘Alice in Wonderland’ feel to these investigators’ efforts – acting on the public’s behalf, searching over hill and dale and among the paperwork of regulatory bodies and drug companies to put together pieces of data that should have been freely available in the first place,” the BMJ editorial chides. “…What is clear from the linked studies is that past failures to ensure proper regulation and registration of clinical trials, and a current culture of haphazard publication and incomplete data disclosure, make the proper analysis of the harms and benefits of common interventions almost impossible for systematic reviewers.”

At the end of the day, the public health implications are troubling and pervasive, an issue that is compounded when one considers that mandatory requirements – as set forth by law – are being ignored. This should serve as a wake-up call to both the NIH and the FDA to restrict grants to researchers who receive government funds, but violate the rules. This may add impetus to industry to ensure proper and timely publication, as well.

As Harlan Krumolz, a Yale University cardiologist who co-authored one of the BMJ papers, writes in his CardioExchange blog, “the era of missing data must end.”

Source