Avi Baskin: Dr. Chang, really a privilege to be here and excited to share the work that we were able to accomplish during my fellowship at Vanderbilt. So I'm excited to share a little bit of the work that we did looking at the readability of patient-facing summaries on ClinicalTrials.gov for urologic oncology trials. And the idea came to mind when I was a fellow at Vanderbilt because many patients come to Vanderbilt for clinical trials, and the question is, one, how do they know what clinical trials Vanderbilt has? And two, how do they know they potentially could enroll in them? And one of the things that we saw anecdotally was that some of these clinical trial explanations online when you go to ClinicalTrials.gov, which is kind of the gold standard of where trials are supposed to be registered, are very confusing and hard to read, and even they're confusing for clinicians, so I think for patients it's also a problem. So clinical trials, we know, are kind of a gold standard of cancer care, yet only a small percent, 8%, of patients are actually participating in them. And part of that is due to the difficulty understanding the trial information, what the criteria are, if they would actually be a candidate. The AMA and NIH recommend now, and have recommended for some time, that patient-facing materials in general be written at an 8th-grade level, meaning so that anyone can understand them, and then prior analysis looking at content on ClinicalTrials.gov found that it's very difficult to read, long sentences, heavy medical jargon, lots of abbreviations. And we were wondering, specifically within urologic oncology, how this stacks up.
So in 2017, NIH issued specific guidance asking people who were publishing these clinical trials to basically write in plain language, improve it so the public can understand what's going on. And now that was kind of put out in 2017. So the aim of our study is to look at the readability of public-facing urologic oncology trials summaries on ClinicalTrials.gov and to evaluate if there's any changes over time and differences by study sponsor. All of our data comes from the registry on ClinicalTrials.gov, and what we found was for over 17,000 urologic oncology trials, and I'll show how we found those trials, we looked at the brief summary, and the brief summary is basically the first patient-facing description. We looked at different readability metrics, basically say, "How hard is it to read something? At what grade level can someone understand this?" And there's a variety of them that are in the literature. And we looked at this by time and by sponsor type. This is just an example of a trial I pulled up recently. No connection to the trial, but if you're a patient going on ClinicalTrials.gov, let's say you have prostate cancer and it's come back, you want to search what clinical trials are available, this is what you would see. Again, I have no connection with this trial. It just was one of the first that popped up. You have a study overview, you can see on here "brief summary", and this brief summary is supposed to be patient-facing, describing, at a basic level, what's going on with this clinical trial so patients can understand if they'd be a candidate and how to get involved.
So the way that we identified these studies is we had a broad list of search terms related to urologic cancers, prostate cancer, bladder cancer, and we searched for this on the ClinicalTrials.gov website, filtered it down, and then filtered by cancer type. And that's how we ended up with that 17,000, most of them being prostate and kidney, and then bladder, testicular, and penile cancer were more rare. In terms of our results, we basically found that readability was very poor. In fact, that's one of the one things I want you to take away. So this is a graph here. On the x-axis is different readability scores, and then on the y-axis is, "What grade level do you have to be to understand this?" And if you look right here, what you can end up seeing is that you basically have to be a graduate-level student on all of these different readability score metrics to really understand what's going on with these clinical trial brief summaries.
Again, the first patient-facing summary that you have. We looked at trends over time. So if you remember, I mentioned in 2017 the NIH said, "Hey, researchers, clinicians, please make these trials a little bit easier to understand." And we were fine. Over time, the summaries became more complex, looking from the year 2000 to the year 2016. After this, it actually did show a modest improvement, but it's still far above the recommended level of complexity. And finally, we looked at sponsor differences, so between academic, government entity, and industry. So government sponsors were the most readable and industry sponsors were the least readable and used the most abbreviations. Overall, for public-facing urologic oncology trial summaries on ClinicalTrials.gov, they're written at a graduate-level reading standard, far above the recommended level of 8th grade.
The plain-language guidance in 2017 was associated with modest improvement, but overall the readability remains poor and the sponsor type seems to matter. Poor readability represents a modifiable barrier to informed decision-making and clinical trial participation. So overall, take-home message is we're doing a lot of great clinical trials. There's a lot of great industry-sponsored stuff. There's a lot of stuff coming out from the government, coming out from different institutions, but we have to make sure, especially in this era, to be able to transmit our hypotheses, transmit our studies to the general public, to make sure they understand and can get on board. Thank you.
Sam Chang: Avi, that was actually a little bit disheartening. I mean, a great presentation, but you would think that we would do a better job. Most striking figure to me was that the vast majority of whatever way you wanted to determine readability, basically every test said, "These are graduate-level explanations." And this is the first thing. So two questions then that, honestly, I think get to the heart of the matter is, one, how do we better inform patients that there are trials available? I think that's the first barrier of, it is one thing to come to an academic center that has had a history of enrolling patients in whatever disease spectrum they may have in urology, in oncology, in surgery, whatever, but how do we make it more obvious to the public, in general, that there are clinical trials and the role of clinical trials? What do you think we can do in terms of increasing acceptance, or at least knowledge, widespread about it?
Avi Baskin: Yeah. Personally, I think the biggest thing is trying to help patients understand what the concept of a clinical trial is. I think a lot of patients worry that when they go on a clinical trial, they're getting an untested medication, they're becoming some sort of experimental guinea pig, not really, they're being put out to dry, it's something that they're testing on them and they're scared about the prospect. And I think just letting people know that clinical trials are actually the standard of care. Oftentimes when you get on a clinical trial, you're getting either something new or you're getting standard of care. Because of the way a lot of them are built, half of the trial is getting standard of care. So you're getting the current standard of care. I actually think that clinical trials give you an opportunity to get something really closely monitored and with the potential of getting something that's better. And if it's not, at least you know you got the current standard. And that's how I usually try to at least sell it to patients. I say, "The government, whether that's the FDA, the AMA, whatever, they have a lot of regulations about having to prove and follow these patients to prove that their treatment works." So I think it's a way to sell the study, but it's hard, right? Because I think when I've talked to patients and try to enroll them in some of the trials we have here, patients really want to be on a trial because they want to get a new medication, and if they get randomized to not that, they may want to withdraw. And those are things we have to consider. But I think the first step is educating patients and letting them know that the clinical trials are really the standard of care. You really are getting what is the standard of care. And then maybe there's something new that works better, works worse, we don't know, but that you're at the leading standard, at that point.
Sam Chang: Then with that directive from the NIH in 2017, what's the shtick behind that? I mean, here's this directive saying that basically patient-facing material specifically about clinical trials needs to be written at a language level of an 8th grader in terms of understanding what's being studied, what the treatment is, what are the possible costs, all those types of things. So a directive comes out. Maybe a little smidge of improvement, but really not much. So is there any teeth behind it, first? And secondly, if there's not, what can we do as clinicians? Do we push the industry? Do we come up with our own plain-language version that we give individually to patients? It still doesn't, though, help the population as a whole. So tell me your thoughts behind that.
Avi Baskin: Yeah. So I don't think there's much teeth. So I don't think there's much enforcement. That's the first thing. I think a lot of this is institutional, so that's where I think a lot of the drive will come from. For instance, when I'm submitting IRBs for research projects at UC Irvine, a lot of it does require now a lay person, non-medical speak explanation of different parts of the study, especially on the consent form. And I think that's very, very important. I think at an institutional level, and I haven't been involved in this at all, but a lot of institutions will have a website where, basically, you can look up, by disease type, what clinical trials might be available. And making those really patient-friendly would be very, I think, helpful. Artificial intelligence in healthcare in certain areas like AI scribes and documentation has really been excellent and there's a lot of excitement around that, and this could be one of those areas. Because one of the things that ChatGPT or Gemini or whatever you may use is really good at, in my opinion, is it can summarize pretty well. Like, you can give it a document and it can summarize it down. And so stuff like this is if you give it a clinical trial protocol and say, "Hey, can you give a 10-sentence summary of this at the level of a 5th grader?" It would be pretty good at doing that. And it's not that I think clinicians and people running clinical trials don't want to make it accessible to everybody and they don't want to not provide these, but they're really worried about requirements A, B, D, E, F, G of all of the protocols and making sure everything is run perfectly and all the safety events are reported and all of the crucial stuff, the regulatory stuff, procedural stuff that goes into clinical trial, they're working so hard on doing that I think that the patient-facing summary probably is falling to the wayside. It's not that they don't want to do it, but they just have so many other regulatory things that they need to accomplish. So if we can utilize AI to help with these summaries and help patients understand in general, I think it's going to be something that potentially could make a difference.
Sam Chang: No, I think your point is right in terms of, I think in the present-day version, just as you said, AI, if you give it basically a set compilation of material, "Here you go," it does seem to do a good job of being accurate, but being able to then synthesize it in a shorter version, condensed version, an easier-to-understand version, which you can see how would be really helpful. Because some of these trials, some of these medications, some of these things that we are trying to test to try to improve either diagnostic or therapeutic care, really has, sometimes, subtleties that are difficult to understand, and to the point of an everyday individual trying to determine should I do this or should I not? Having something that's easily understandable and makes sense is only beneficial for everyone. For the trial investigator, obviously for the patient, for the population as a whole. So by your work, I think it really at least elevates the idea or the importance for trial investigators like myself, we've got to make these understandable. We've got to make these actually something that can be interpreted by the population as a whole and then up to the individual in their situation regarding, "Should I do this trial or not?" And your point regarding the fact that there are viable standards of care that are going to be provided to the patient, in addition, sometimes, and actually the majority of times, these patients are even followed more closely, more rigorously on a schedule that to be followed, etc. So I really applaud all the efforts that you're making, Avi. What next? Are you going to start approaching the industry companies with their trials? Where are you going to go next?
Avi Baskin: Yeah, one of the things that I thought about would be a way to make, even for me starting on faculty at UC Irvine, one of my first questions was, "I know UC Irvine has pretty robust clinical trials within GU oncology, for instance," but I was like, "what are they? What's going on right now? Okay, for bladder, what do we have? What's open? What's closed?" Asking for each of the disease spaces? And even as a urologic oncologist, and I am new in my career, so that's a caveat, but it was confusing coming in to know what I could offer patients. And so if it's confusing for me, it's going to be confusing for the patients too. And I think a way to simplify that down and find people who would be candidates for these trials, you could kind of build something out a little bit to make it a little bit more, "Oh, a clinician knows to enroll people, so they do." Whereas the next person tells them... Like, we had a clinical trial that's being run here, a small study just looking at MRIs in the pre-resection space for a person with a new bladder mass. And I just didn't know about it. I just didn't know. I was new. And they were like, "Oh, have you had anyone for this trial?" I was like, "I'm sorry, I didn't know about it. I've probably had 10 people for your trial." And I think building systems to get people to know about the trials, both from a patient perspective, but you should be able to go to a website and a patient says, "Hey, I have prostate cancer. These are the details," where I upload a PDF of their prior note. "Does this institution have any clinical trials?" And it could kind of try to match you. So those sorts of things to make the process easier would be exciting next steps to take.
Sam Chang: Yeah. No, I think that's great. And it would be great if we could standardize that between institutions, et cetera. Maybe it's through the electronic medical record, maybe it's through something just so that having that accessibility from, if you're in Nashville, Tennessee, or if you're out in Southern California or if you're in North Dakota, just to have an idea of just a balance of what's out there and what's available. So Dr. Baskin, Avi, just fantastic to spend some time with you. And as always, we look forward to your fertile mind as it comes up with other ideas to improve patient care, patient diagnosis, patient therapeutics, and so we look forward to talking to you again.
Avi Baskin: Really appreciate you having me on and always good to see you. So thank you for the opportunity.