It has been 20 years since UK doctors were instructed not to prescribe the antidepressant paroxetine – sold as Seroxat in the UK and Paxil in the US – to adolescents. The instruction came after a report showed it could be linked to suicidal ideations, with reanalysis of the data found that the benefits for young people were far less and the harmful effects far greater than the study suggested.
There has been plenty of development with trial reporting since. In 2013, the Pharmaceutical Research and Manufacturers of America (PhRMA) and the European Federation of Pharmaceutical Industries and Associations (EFPIA) recognised the importance of data sharing and supported initiatives to enhance clinical trial data transparency and promote scientific advancements.
In 2017, the European Medicines Agency (EMA) and US Food and Drug Administration (FDA) made some changes to clinical trial legislation which included changes about which trials needed to report data publicly. Following this, Dr Nicholas DeVito, who worked on physician and author Dr Ben Goldacre’s team at Oxford University, was tasked by his boss to create a trials tracker for both the EU and US which would keep an eye on which trials reported the data as required by the EMA and FDA.
DeVito says there has been a significant improvement since the trackers were published five years ago but there is still more that needs to be done to ensure sponsors are not only reporting data when needed but that the data is also accurate and meaningful.
Abigail Beaney: How did the trial trackers come about and what is their purpose?
Dr Nicholas DeVito: In 2017, there were some very recent changes to legislation and implementation of legislation in both the US and the EU. That gave Dr Ben Goldacre the idea that we could automate the checking of whether companies who should have reported their results did so under various regulations and laws in both jurisdictions. So, he tasked me with doing that. It uses both the US registry, ClinicalTrials.gov, and the EU Clinical Trials Registry and we had to understand how we could identify which trials were due to report results, how we could automatically ingest and analyse that data, and operationalise that into these live tracking websites. The websites have been designed as a tool that can be used for both transparency and accountability, but also for the sponsors themselves, who are able to see their own performances and hopefully let them more easily address any missed data.
AB: Do you think that the trial trackers have made sponsors feel like they should be more accountable in reporting trial data?
ND: People like to talk about the trials trackers as a naming and shaming tool and while it certainly is not a bad thing that people who do poorly on clinical trial reporting look poorly on our website, that wasn't the primary motivation.
A big point of this was building a tool that could be curated by a sponsor to see their portfolio and address it and then they can see it get better because it is live updating. The feedback we have had has certainly made it feel like it has had an impact. We have had contact with most of the major pharma companies where they have someone whose job it is to check and make sure they are hovering up around that 100% mark and are in full compliance.
Big Pharma has been quite good at reporting – they might moan when the rules get proposed but once they get implemented, they tend to be quite good at following them. Sometimes, however, the quality of what’s reported might be a different story. Then academic sponsors – we have got lots of feedback from academic sponsors, but I don't know that it's been quite as widely adapted there. In the UK, we actually got political attention around the tracker and it was like night and day from around 2018 to around 2020, things changed very rapidly.
AB: Have you found that there's any reason why sponsors would avoid reporting its data?
ND: That’s the motivating factor of why we had the registries and the whole infrastructure. For example, the paroxetine scandal back in the early 2000s and the suppression of data leading directly to harm. I think we were always waiting for the next instance of something like that to happen, but these laws are meant to prevent that. I think that nefarious suppression of data is probably the rarer instance in the modern world, I would say nefarious intent would be the exception rather than the rule. I think sponsors wanting to suppress results forever is rare, but they can drag their feet in ways that are beneficial to them in one way or another. The motivations behind any late reporting of course we don't know but I think the vast majority of unreported trials right are pretty innocent oversights. That could be where they missed one on a list somewhere or it could be that the trial was really old and the person who did it died or left the institution, and they cannot get in touch with them anymore and the data is gone. Then there's also lack of awareness, some sponsors don't know that they have the responsibilities to report. In the UK, our tracker led to letters sent by Parliament which really raised awareness.
AB: We're at a time where public trust in the medical industry and pharma is really quite low. Do you think that the lack of reporting emphasises this in that the public don't believe that the industry is being completely transparent?
ND: Yeah, reporting has got much better and is overall quite good, right. The best estimates we had from the early 2000s and even the 2010s was that around 50% of trials remain unreported. These days you're seeing around 75% of trials reported. While any trial being unreported has the potential to cause trust issues with the public, you can't take reporting to just be this binary thing. Sponsors can report all the results, but they can do things in how they are reporting and the quality of that reporting and use strategies in how they analyse and report their data that are less than scrupulous. When that becomes unearthed it can cause just as much harm to public trust.
AB: What ways are there that sort of sponsors can misuse data?
ND: You can use real data, you're not making up data, but you can analyse it or interpret it in ways. Outcome switching is a big issue. That can happen legitimately, for example, the devices malfunction, and the sponsor cannot measure the primary outcome. That is not nefarious but there's lots of undisclosed outcome switching, it is highly prevalent. Sponsors can also choose weird endpoints, even if they are reporting the one they said, they can choose endpoints that are more likely to be more favourable, especially trying to get like a drug approved in the US.
There's a lot of controversy in pharma around Alzheimer's medicine and the endpoint of amyloid reduction and whether that's an appropriate proxy for clinical benefit. It's still sort of an open science question as to whether that actually leads to any benefit as a proxy outcome. Some oncologists talk about progression-free survival as an oncology outcome versus overall survival, which matters more like did the patient die rather than did their tumour shrink - that may or may not be clinically meaningful. These are accepted endpoints; they've made their way into general practice and made their way to be accepted by regulatory bodies. Having treatments where we're not actually sure that they work or not based on outcomes like this getting approved. I don't think that's doing much for faith either.
AB: What would be the answer to this? Would it be better to have standardisation of trial endpoints and standardisation in which the way that data is collected and reported?
ND: Something like that could work. You would need some sort of regulatory or international standard and someone like the FDA to say they will no longer accept this as a standalone primary outcome method by which to approve things. There are pressures that prevent that from happening and there are very good reasons why we ended up in the situation we're in now. Certain outcomes that are used as primary outcomes are beneficial for sponsors who want their drugs approved and the FDA is very reliant in their funding on approving drugs. As a result, there's this perverse incentive on approving drugs based on substandard data and there's these accelerated pathways that require less stringent evidence that should probably be revisited and tightened up to ensure we're truly getting effective and safe medicines on the market. People raise concerns about it and then those concerns trickle into the mainstream media and then when people see that, even if they don't fully interpret what's going on, they think how can I trust anything coming out of pharma?
