Kevin Gallagher: Thank you very much, Ashish. It's my pleasure to present to you today about this study on behalf of 13,000 collaborators that took part. So, RESECT was a randomized control trial of audit and feedback in TURBT surgery. We wanted to ask the question, a fundamental question about whether having audit and feedback as a surgeon, as a bladder cancer surgeon, can change how you behave and actually improve outcomes for patients. So, that was the aim of the study. And the way it was designed was that it was a cluster-randomized controlled trial. It involved 220 hospitals and more than 15,000 TURBT cases. They were first tumor resections only, where we presumed it was non-muscle-invasive bladder cancer. It involved the collection of data, retrospective data, so that we got a baseline for all of the sites. What was it like before they were involved in this study? And then sites collected data prospectively going forward. The intervention, so after randomization sites were randomized, half the sites were given an intervention. And the intervention was to receive a feedback dashboard about your performance. So, that would compare you to your peers, give you guideline statements about what the guidelines recommend, and provide a little bit of education. Were given access to that, or the control arm was not. They just had usual practice, went on collecting data and submitting it for the study, but had no feedback, no comparison. And the outcomes were, the primary outcomes were four TURBT quality indicators.
I'll just list those on the next slide. And a key secondary outcome was the rate of recurrence at the first check cystoscopy. The outcomes we looked at, the quality indicators was the rate of detrusor muscle sampling, the rate of single instillation intravesical chemotherapy use, documentation of the completeness of resection, so making a firm statement that I have microscopically cleared all of the tumor and writing that in the record, and then clear documentation of all relevant tumor features. And as mentioned, the key secondary outcome recurrence at the first check. We also assessed this idea of a Hawthorne effect, which is the change that happened in the control group between before and after. So, what changed just through the knowledge that you were being watched. And a quick summary of what we found. So, the intervention, so just receiving feedback about your performance significantly improved the documentation of tumor features over and above a little bit of improvement that was seen in the control arm, and similarly for documentation of the completeness of resection. We didn't find that the intervention improved single instillation chemotherapy rates over and above what was seen in the control arm, but importantly, across the study we did see a trend to increased use of single instillation intravesical chemotherapy compared to what was occurring prior to the study. So, the control arm study giving more chemotherapy, and therefore it was hard for the intervention group to improve more than that.
For detrusor muscle resection, we saw no effect of either the Hawthorne effect or the intervention, but interestingly, at baseline detrusor muscle resection rates were pretty high, so about 80% across the board, we think there was a bit of a ceiling effect here. With regards to recurrence rates, so if we look at the effect of the intervention, so did the intervention group improve more than the control group? And the answer is no, there was no difference between control and intervention. But if we look at the Hawthorne effect, and importantly, this was adjusted for confounders really robustly in over 2,000 patients before the study and in study groups. We saw a significant reduction in the early recurrence rate, from 24% to 18%, and that was highly significant on the adjusted model. So, in conclusion, providing private performance feedback and peer comparison improves some behaviors, some surgical practice in TURBT surgery, but we find a powerful effect of being watched. And that translated to a 25% relative reduction in recurrence from before versus during the study. And we think this is really important, really significant, and indicates that routine audit and feedback should be recommended in those performing TURBT surgery. Thanks. We want to say a particular thank you to the huge group of worldwide collaborators that took part in this, and also a special thank you to Param Mariappan, whose baseline work in Scotland a lot of this is based on. Thank you.
Ashish Kamat: Thanks, Kevin, for that wonderful presentation. Really like the imagery in your slides as well. Highly polished. Sort of reminds me of the transition from the old SD monitors to now the high-definition, blue light, state-of-the-art monitors. I mean, some of the background images, really nice. So, were you surprised? Bottom line, were you surprised by your own findings?
Kevin Gallagher: I always said when we were doing this that I think it's a really hard bar, because for the intervention to work it's got to be even better than what happens in the control arm. So, I think if you know you're being watched, you try a bit harder. And we did see that in the control arm, but the intervention improved some things even more. So, there's a benefit from knowing you're being watched, but there's also a benefit from having structured feedback and peer comparison. That also helped.
Ashish Kamat: Yeah, you and I have chatted about this over the years. I think one of the early feedbacks that we shared was there were some centers and places when they found out that they were being watched, not just anonymously, but by people who knew who they were, almost didn't want to participate. So, that fear of being watched can have a positive effect. We hope it has a positive effect in our field in general. But it can also have a negative impact. I always think audit self-reflection is very important. Share with us some of your insights maybe through this trial, but even in general, the importance of feedback, immediate and long-term, early delayed on something quite as what we consider a routine, but important as a TURBT.
Veeru Kasivisvanathan: Yeah, so I consider audit and feedback of our practice one of the cornerstones of what we do. We train to do this procedure, which is how we diagnose and treat the tumor, but actually how good are we doing? And in the UK and in many other centers there'll be this need to try and audit your own outcomes and see how good you are. The question is whether that's actually being done. So, RESECT was a really good way of allowing centers to take part in that process in a means that was easy for them. And I guess one of my reflections was, as you said, some centers didn't want to take part, because they're being watched, but we designed this specifically to be non-punitive. So, every surgeon on that comparison graph where they could see how well they did for the quality indicators against other centers was anonymized. You could see where you were compared to the others, but you didn't know who the others were. And we wanted to ensure that it was non-punitive because of the nature of the study, and we wanted to get engagement.
So, on the whole I would say more than 95% of centers took part and didn't have a problem. But I would say, we might have seen even better results had we not anonymized. And an example I'll give you is the Scottish QPI study led by Param Mariappan where they do publish their data stating who the centers are and who the teams are, and their results have shown slight improvement in various factors over time. So, I think there is the potential for, or even more, when you know who the surgeons are, you know you're being watched, that feedback can influence your practice even more. But despite that, I would say what we did see with the anonymized approach was an improvement in achievement of two of the four quality indicators. And I think you mentioned, did the results surprise you? I would say the extent of the Hawthorne effect, the control group improving so much with a reduction in the recurrence rate by 25%, for me, that was quite big and probably a bit surprising. So, it can be done even with the anonymized feedback.
Ashish Kamat: Yeah, no, great points. I think one of the things I want to highlight from what you said in case people don't pick up on it, is the Hawthorne effect. And the Hawthorne effect, again, it's a little bit comical, a 1920s study of light bulbs essentially and workers and changing the light bulb on a Sunday and then attributing that to increased productivity on a Monday and a whole bunch of other things. But I think the key thing here is that in our field, in what we do, there are so much advances when it comes to systemic therapy for bladder cancer that really, truly affects about 20% of patients. There's so much advances and cost invested into BCG-unresponsive disease. Clearly we need it, but affects a small percentage of patients. A TURBT in general almost affects 95% of patients. Maybe 5% of patients presenting we do know a metastatic disease may not ever need a TURBT, but the vast majority of patients across the globe need this critical procedure. And over the years it's been relegated in many systems across the globe to the junior-most person or the least experienced faculty.
And I think this data seems to suggest that you can improve. At whatever level you are, junior, senior, experienced, non-experienced, retiring or just starting out, you can improve and that improvement actually will benefit our patients. I think that is a huge win, not just for you guys. I mean, kudos to you all for embarking and doing this massive effort. But I think for the patients in a whole, and I think this feedback loop should ... We incorporated here for our training and our trainees, I think it should be incorporated in practice as well, not just for trainees, but active surgeons. And with that little statement, question to you, are you planning to take this into a loop or some sort of an AI-based effort where you could incorporate this into EHRs, EMRs, training programs?
Veeru Kasivisvanathan: Great question. And to me, the results of this study suggests we should be doing national, not just local, but national audit of TURBT practice, which mean, like for example, you mentioned outcomes in bladder cancer and progression and how many patients this affects. If you take a similar cancer like prostate cancer, patients do a little better with prostate cancer, but we have so many national audits of practice in that, they're all funded nationally from a government level. If we were to think about bladder cancer where we know your technique makes such a big difference to your outcomes, I would say this evidence supports national audits of TURBT practice across the country, and I would be pushing for that locally and with the guidelines committees.
Ashish Kamat: No, that's great. And one of the efforts that we've led through the International Bladder Cancer Group, and the one that we're focusing on this year is actually TURBT, improving quality, improving standard practices. And Param, who's a member of the IBCG has been tasked with leading that. And I'd love to have the two of you join our efforts as well if you're interested. In the interest of time we're going to wrap it up, but I do want to leave both of you with a closing word. So Kevin, some closing thoughts for our audience, and then Veeru?
Kevin Gallagher: I think this study shows how powerful just knowing your own outcomes and your own performance compared to your peers can be in a non-punitive, anonymous way, and we should seek to embed that in all of our practice, whether it's bladder cancer or other urological surgeries, it will benefit patients.
Veeru Kasivisvanathan: Yeah. And I might add, this massive effort, it was because of the interest from surgeons in wanting to improve that we managed to achieve what we did in this study. And it just shows me that there is a willingness to do this. And if you're provided with the tools to do it, it lowers that threshold. So, I think we should focus on lowering the threshold for everyone in the world who does TURBT for 95% of patients diagnosed with bladder cancer to help them improve their outcomes on a patient level.
Ashish Kamat: Absolutely. Well said, both of you. Thank you for taking the time, and congratulations once again.
Kevin Gallagher: Thank you very much.
Veeru Kasivisvanathan: Thank you.