Mark Hennon, MD
There is an ever-increasing push towards transparency in the healthcare industry with public reporting of outcomes through various ranking systems. Criteria used to determine rankings of academic and community programs can vary widely.
As an example, US programs participating in the Society of Thoracic Surgeons General Thoracic Database are evaluated based on perioperative outcomes for lobectomy. However, how short-term surgical outcomes relate to long-term survival for patients undergoing resection for non-small cell lung cancer (NSCLC) is not clear. With that in mind, we recently analyzed results from the large multi-institution National Cancer Database (NCDB) to assess for any correlation between short-term and long-term outcomes for programs treating patients with NSCLC.1
By creating a scoring system for analyzing both short- and long-term outcomes, we were able to rank programs into 10 groups based on outcomes for patients undergoing surgery, as well as rank programs for overall survival for all patients undergoing treatment for NSCLC. By plotting the ranking of any individual program for each metric against itself, one could determine if any correlation between the two rankings for programs is apparent.
As seen in Fig. 1, there is no consistent correlation when analyzing programs that treated more than 200 patients per year. It is worthwhile to mention that the short- and long-term metrics created in our analysis were not done with the intent to propose a comprehensive quality metric. Such an effort will need more granular data for risk adjustment as well as extensive validation. The intent was to test the assumption that centers with excellent short-term outcomes also had excellent long-term outcomes.
Clearly, such an assumption is not true.
Sai Yendamuri, MD
Though there are known limitations associated with the NCDB dataset, its strength is the opportunity it afforded us to analyze patients undergoing surgery for NSCLC as well as those patients not undergoing surgery at a specific institution or program. Though imperfect in its level of granularity pertaining to comorbidity data and its lack of disease-free survival data, it provides a reasonable starting point for trying to answer the question of whether proposed outcomes-reporting mechanisms offer a complete picture for patients. Commentaries have offered differing opinions, including highlighting the limitations of the analysis.2,3
One way to think about our results is to examine them through the lens of the law of unintended consequences—when the solution to the problem is worse than the problem itself. If the problem is a lack of transparency and useful data for patients attempting to make informed healthcare decisions, and the solution is public reporting of outcomes for patients, we should strive to ensure the data available to patients offers the most complete picture possible. One could envision a scenario where public reporting of short-term surgical outcomes leads to increased risk aversion for surgery, which is arguably an unwanted consequence.
Without question, increased transparency for patients making healthcare decisions is a positive development and is much needed. However, it behooves us to provide a complete picture to patients desiring such information. Our study highlights the existing lacuna in this space.
References
- 1. Hennon M, Groman, Kumar A, Castaldo L. George S, Demmy T, et al. Correlation between perioperative outcomes and long-term survival for non-small lung cancer treated at major centers. J Thorac Cardiovasc Surg. 2022;163:265-273.
- 2. Raymond DP. Commentary: Understanding limitations. J Thorac Cardiovasc Surg. 2022;163:274-275.
- 3. Scarci M, Raveglia F. Commentary: Waiting is among the great arts (or rather, why oncologic programs should be rated on long-term outcomes). J Thorac Cardiovasc Surg. 2022;163:275-276.
