CMS Star Ratings Unreliable for Surgical Quality?

—
Higher ratings tied to better surgical outcomes, but variability was high even within star groups

by
Michael DePeau-Wilson,

Enterprise & Investigative Writer, MedPage Today

June 18, 2024

Higher CMS hospital star ratings generally were associated with better surgical outcomes, but there was wide variability within each star-rating group, suggesting its utility for assessing surgical quality may be limited, according to a cohort study.

In an analysis of 3,240 CMS star-rated hospitals, 30-day mortality rates fell from 6.80% for 1-star hospitals to 4.93% for 5-star hospitals (adjusted OR 1.86, 95% CI 1.73-2.00), and similar trends were observed for complication rates, serious complication rates, and 30-day readmission, Adrian Diaz, MD, MPH, of The Ohio State University, and colleagues reported in JAMA Surgery.

However, there was wide variation in these outcomes when comparing the top and bottom 5% of hospitals — even within star-ratings groups. For instance, mortality varied nearly two-fold for each group:

• 1 star: 1.91% (3.6%-12%)
• 2 star: 1.86% (2.8%-16.2%)
• 3 star: 1.84% (2.9%-12.3%)
• 4 star: 1.76% (2.9%-11.5%)
• 5 star: 1.79% (2.4%-9.1%)

Once again, similar trends in variation were observed for complications, serious complications, and readmissions, they reported.

“The star rating system is publicly available [and] it’s widely used, but it’s probably not the best tool for patients to pick the hospital to have surgery or to pick the surgeon that’s going to do their operation,” Diaz told MedPage Today. “It’s just not robust or specific enough for that.”

CMS first released its hospital star ratings program in 2016 to establish a public database of hospital quality. It was met with controversy, however, as experts cited methodologic limitations. The program was paused in 2020 for evaluation, and a revised version was released in 2021.

Diaz said this study is the first to evaluate surgery-specific outcomes with the revised ratings.

Many 5-star hospitals had higher rates of adverse post-surgical outcomes than hospitals with lower star ratings, even in the same geographic regions, Diaz noted. That suggests the CMS star rating system may not adequately inform patients about the quality of specific operations at any given hospital, he said.

Publicly available procedure-level data could help patients decide which hospital to choose for their specific health needs, Diaz added.

“[Patients] should be able to hit the search button … and the data should be able to tell [them] these are your options. These are those surgeons’ and those hospitals’ outcomes, and based on your preferences, these are your top three choices,” Diaz said. “Nothing like that exists for patients to make those decisions, but I think the data is available and we have the tools to make that possible.”

In an invited commentary, Anoosha Moturu, MD, of the American College of Surgeons (ACS), and colleagues noted that evaluating surgical quality is difficult for many reasons, including the fact that the “source of data most used for evaluating quality is suboptimal. The CMS and other benchmarking organizations use claims data … for many of their metrics, which has been shown to be inferior to clinical data for reporting surgical outcomes.”

They added that ACS has started to address these deficiencies by developing a public reporting site and compiling metrics that are “based on lessons from decades of leading surgical quality of care measurement, standards development, accreditation, and improvement.”

For their analysis, Diaz and colleagues analyzed data on 1,898,829 Medicare beneficiaries who had either colectomy, coronary artery bypass graft, cholecystectomy, appendectomy, or incisional hernia repair at 3,240 nonfederal acute care hospitals with a CMS hospital star rating from 2014 to 2018.

Overall, patients had a mean age of 74.8 years, 50.6% were male, and 86.5% identified as white.

The study was limited by only including Medicare patients, who tend to be older and sicker than the general population. Also, the dataset did not include any patients on Medicare Advantage plans, further limiting its generalizability.

Diaz urged clinicians to “write to CMS, write to their representatives,” to “get this stuff out there.”

“The reason the rating system was revised over a year ago was because of providers like us — and others around the country — who have demonstrated this is an imperfect system,” he said. “It’s continued to be imperfect, but there’s ways that we can strive to make it better.”