This transcript has been edited for clarity.
Hello, everyone. This is Dr Bishal Gyawali from Queen's University, Kingston, Canada. Welcome back to the Skills Lab series on Medscape. Today we are continuing with how to read a phase 3 clinical trial report. As you may have watched in the previous four videos, we talked about some background information even before we started reading the clinical trial report. In the last video, I talked about the abstract, focusing on when and why to read it.
Today, continuing with that same series of how to read a phase 3 clinical trial report, the introduction comes after the abstract. Let's talk about what to look out for in the introduction.
The introduction is the section that sets the background for why the trial was done. There are just a couple of things that I want to look at in the introduction section. One is the biological rationale for the drug. Especially if it's a new drug, I want to know what the biological rationale was for doing this trial. Does it make sense biologically?
The second one, and the most important thing that I want to look for in the introduction, especially for a phase 3 clinical trial, is whether a phase 1 or phase 2 trial has been done in the past for the same drug, ideally in the same setting. This is important because we don't want to see a phase 3 trial done without a prior phase 1 or phase 2.
We briefly talked about the different phases of clinical trials in the first video. The role of a phase 1 trial is to establish safety, and I'd assume that almost all the new cancer drug trials will have undergone a phase 1 trial.
Phase 2 is tricky. Nowadays, we have seen several trials jump from phase 1 directly to phase 3 without undergoing phase 2. Why does that matter? If you look at data of 100 preclinical studies that happen, only 5% go on to receive approval. The biggest risk of failure would be at a phase 3, and the biggest investment from a drug development point of view also is at the phase 3 part of the trial in the drug development process.
A typical phase 3 would cost somewhere around $22 million to conduct whereas a phase 2 would cost around $11 million to conduct. We don't want, ideally, a drug to fail at phase 3. We want better screening at phase 2 so that drugs that are more likely to be successful at phase 3 are screened during phase 2.
There have been a couple of studies looking into this. One of these studies showed that, in a phase 2 trial, the objective response that we see in a phase 2 seems to be a useful endpoint for screening new targeted agents because it predicts eventual success in phase 3. That is the role of the objective response rate in a phase 2 trial, whether or not that drug will be successful at phase 3 rather than approving drugs based on the phase 2 response rate itself.
Similarly, in one of the analyses that we did about the phase 3 trials that failed, we saw that, for 42% of the trials that failed at phase 3, a phase 2 trial was not available. Of those trials where phase 2 was available, 28% of them had failed in phase 2. Despite the drug failing at phase 2, it was still tested in a phase 3, and of course, that phase 3 failed as well.
For every phase 3 trial, in the introduction, we want to check whether or not a previous phase 2 was done, and whether that phase 2 was successful, or whether the phase 3 trial was conducted despite phase 2 failing, in which case, we would assume that the phase 3 was done just to chase a statistical significance and get the drug approved without passing through the phase 2.
That's all about the introduction. Then we want to move on to the methods section. In the methods section, we'll see different subsections that are reported. As I keep saying, the methods section is the most important to understand the trial, but it is the least-read section. I want to encourage all of you to actually go through the methods section in detail.
I'll tell you what to look at in the methods section. The first paragraph of a methods section ideally talks about what types of patients were included, which means the inclusion and the exclusion criteria for the trial. This is very important because we want to make sure that the patients who are enrolled in the trial are representative of patients that we see in the daily clinic.
That means we want to see diversity in patients that are enrolled in these clinical trials. For example, in terms of gender diversity, what was the male/female participation; in terms of age diversity, whether elderly patients were included; in terms of race and ethnicity diversity, what ethnicity or race of patients have been included; in terms of diversity in performance status, are patients with performance status of 2 included? We want to see this representation in the clinical trials.
In one of the studies that we published almost 4 years ago now, we looked at the phase 3 trials of cancer drugs that supported the FDA approval of these drugs. We compared the patient population with that of the national representative sample by using the SEER database.
We saw that female patients were systematically underrepresented in the pivotal phase 3 trials, only 36% vs almost 50% in the general database. Black patients were underrepresented, only 2.1% vs 9.8% in the database. Patients with hepatitis B infections, hepatitis C virus infections, and brain metastases have been systematically underrepresented in pivotal clinical trials.
There have been some efforts in this regard to broaden and expand the eligibility criteria. ASCO has produced a statement saying that patients should not be systematically excluded based on, for example, their brain metastases, age group, HIV infection status, or renal function criteria. We still have a long way to go.
Reading eligibility criteria is quite important, and this is important even for interpretation of the trial. I'll give you one example: the trial of sorafenib in advanced liver cancer. Sorafenib was the first targeted drug to show overall survival improvement in liver cancer vs placebo.
If you look at the pivotal trial, you see that patients receiving placebo had a median survival of almost 8 months, and sorafenib had a median survival of almost 11 months. This was a 3-month improvement in survival, which looked fine.
If you look at this study done in the real world, they found that patients who got sorafenib in the real world had shorter survival than patients who got placebo in the trial. Just think about it: Patients getting placebo in the trial had better survival than patients getting sorafenib in the real world. Why does that happen? It is because of the eligibility criteria.
The trial eligibility criteria are so strict that only Olympian patients are eligible. They are patients with cancer, but they are so fit that they are eligible to participate in the trial, so of course their performance will be better and is not reflective of what happens in the real world. It's almost funny. These are patients with cancer in the liver, but one of the eligibility criteria was to have pristine liver function.
After the eligibility criteria, we look into randomization and how the randomization was conducted. Classically, this used to be a big deal, but nowadays, almost all trials use a computerized, standardized randomization system. I have never seen a problem with this in big cancer trials.
The randomization ratio: Classically, we used to see 1:1 randomization, but nowadays we have seen unequal randomization. That means randomization in the ratio of 2:1 or 3:1. We do not have time to go through the details of why unequal randomization is not ideal. I would recommend people to read this article that I have cited here.
The case against unequal randomization is because an unequally randomized trial needs more sample size than a randomized trial, so we need more patients for such a trial. That means it will be more expensive to do such trials and there will be less power to detect the effect size with unequal randomization.
That also means that, because almost half of the phase 3 trials actually fail, we are unnecessarily exposing more patients to that unproven intervention without knowing that it is beneficial for the patients. If we think that the interventional arm is so good that more patients should be randomized to it, then that means there is no clinical equipoise and why are we even doing the trial? Actually, there is no evidence that unequal randomization helps to increase recruitment into the trial.
That's all for today's video. I hope you have found this useful. In the next video, we'll talk more details about methods, so we'll continue with the methods section. Thank you.
COMMENTARY
Skills Lab: Assessing the Introduction and Methods Sections of a Clinical Trial Publication
Bishal Gyawali, MD, PhD
DISCLOSURES
| January 08, 2025This transcript has been edited for clarity.
Hello, everyone. This is Dr Bishal Gyawali from Queen's University, Kingston, Canada. Welcome back to the Skills Lab series on Medscape. Today we are continuing with how to read a phase 3 clinical trial report. As you may have watched in the previous four videos, we talked about some background information even before we started reading the clinical trial report. In the last video, I talked about the abstract, focusing on when and why to read it.
Today, continuing with that same series of how to read a phase 3 clinical trial report, the introduction comes after the abstract. Let's talk about what to look out for in the introduction.
The introduction is the section that sets the background for why the trial was done. There are just a couple of things that I want to look at in the introduction section. One is the biological rationale for the drug. Especially if it's a new drug, I want to know what the biological rationale was for doing this trial. Does it make sense biologically?
The second one, and the most important thing that I want to look for in the introduction, especially for a phase 3 clinical trial, is whether a phase 1 or phase 2 trial has been done in the past for the same drug, ideally in the same setting. This is important because we don't want to see a phase 3 trial done without a prior phase 1 or phase 2.
We briefly talked about the different phases of clinical trials in the first video. The role of a phase 1 trial is to establish safety, and I'd assume that almost all the new cancer drug trials will have undergone a phase 1 trial.
Phase 2 is tricky. Nowadays, we have seen several trials jump from phase 1 directly to phase 3 without undergoing phase 2. Why does that matter? If you look at data of 100 preclinical studies that happen, only 5% go on to receive approval. The biggest risk of failure would be at a phase 3, and the biggest investment from a drug development point of view also is at the phase 3 part of the trial in the drug development process.
A typical phase 3 would cost somewhere around $22 million to conduct whereas a phase 2 would cost around $11 million to conduct. We don't want, ideally, a drug to fail at phase 3. We want better screening at phase 2 so that drugs that are more likely to be successful at phase 3 are screened during phase 2.
There have been a couple of studies looking into this. One of these studies showed that, in a phase 2 trial, the objective response that we see in a phase 2 seems to be a useful endpoint for screening new targeted agents because it predicts eventual success in phase 3. That is the role of the objective response rate in a phase 2 trial, whether or not that drug will be successful at phase 3 rather than approving drugs based on the phase 2 response rate itself.
Similarly, in one of the analyses that we did about the phase 3 trials that failed, we saw that, for 42% of the trials that failed at phase 3, a phase 2 trial was not available. Of those trials where phase 2 was available, 28% of them had failed in phase 2. Despite the drug failing at phase 2, it was still tested in a phase 3, and of course, that phase 3 failed as well.
For every phase 3 trial, in the introduction, we want to check whether or not a previous phase 2 was done, and whether that phase 2 was successful, or whether the phase 3 trial was conducted despite phase 2 failing, in which case, we would assume that the phase 3 was done just to chase a statistical significance and get the drug approved without passing through the phase 2.
That's all about the introduction. Then we want to move on to the methods section. In the methods section, we'll see different subsections that are reported. As I keep saying, the methods section is the most important to understand the trial, but it is the least-read section. I want to encourage all of you to actually go through the methods section in detail.
I'll tell you what to look at in the methods section. The first paragraph of a methods section ideally talks about what types of patients were included, which means the inclusion and the exclusion criteria for the trial. This is very important because we want to make sure that the patients who are enrolled in the trial are representative of patients that we see in the daily clinic.
That means we want to see diversity in patients that are enrolled in these clinical trials. For example, in terms of gender diversity, what was the male/female participation; in terms of age diversity, whether elderly patients were included; in terms of race and ethnicity diversity, what ethnicity or race of patients have been included; in terms of diversity in performance status, are patients with performance status of 2 included? We want to see this representation in the clinical trials.
In one of the studies that we published almost 4 years ago now, we looked at the phase 3 trials of cancer drugs that supported the FDA approval of these drugs. We compared the patient population with that of the national representative sample by using the SEER database.
We saw that female patients were systematically underrepresented in the pivotal phase 3 trials, only 36% vs almost 50% in the general database. Black patients were underrepresented, only 2.1% vs 9.8% in the database. Patients with hepatitis B infections, hepatitis C virus infections, and brain metastases have been systematically underrepresented in pivotal clinical trials.
There have been some efforts in this regard to broaden and expand the eligibility criteria. ASCO has produced a statement saying that patients should not be systematically excluded based on, for example, their brain metastases, age group, HIV infection status, or renal function criteria. We still have a long way to go.
Reading eligibility criteria is quite important, and this is important even for interpretation of the trial. I'll give you one example: the trial of sorafenib in advanced liver cancer. Sorafenib was the first targeted drug to show overall survival improvement in liver cancer vs placebo.
If you look at the pivotal trial, you see that patients receiving placebo had a median survival of almost 8 months, and sorafenib had a median survival of almost 11 months. This was a 3-month improvement in survival, which looked fine.
If you look at this study done in the real world, they found that patients who got sorafenib in the real world had shorter survival than patients who got placebo in the trial. Just think about it: Patients getting placebo in the trial had better survival than patients getting sorafenib in the real world. Why does that happen? It is because of the eligibility criteria.
The trial eligibility criteria are so strict that only Olympian patients are eligible. They are patients with cancer, but they are so fit that they are eligible to participate in the trial, so of course their performance will be better and is not reflective of what happens in the real world. It's almost funny. These are patients with cancer in the liver, but one of the eligibility criteria was to have pristine liver function.
After the eligibility criteria, we look into randomization and how the randomization was conducted. Classically, this used to be a big deal, but nowadays, almost all trials use a computerized, standardized randomization system. I have never seen a problem with this in big cancer trials.
The randomization ratio: Classically, we used to see 1:1 randomization, but nowadays we have seen unequal randomization. That means randomization in the ratio of 2:1 or 3:1. We do not have time to go through the details of why unequal randomization is not ideal. I would recommend people to read this article that I have cited here.
The case against unequal randomization is because an unequally randomized trial needs more sample size than a randomized trial, so we need more patients for such a trial. That means it will be more expensive to do such trials and there will be less power to detect the effect size with unequal randomization.
That also means that, because almost half of the phase 3 trials actually fail, we are unnecessarily exposing more patients to that unproven intervention without knowing that it is beneficial for the patients. If we think that the interventional arm is so good that more patients should be randomized to it, then that means there is no clinical equipoise and why are we even doing the trial? Actually, there is no evidence that unequal randomization helps to increase recruitment into the trial.
That's all for today's video. I hope you have found this useful. In the next video, we'll talk more details about methods, so we'll continue with the methods section. Thank you.
Any views expressed above are the author's own and do not necessarily reflect the views of WebMD or Medscape.
TOP PICKS FOR YOU