With the current focus on social risk factors (SRFs) affecting health care, it is not surprising that methods for comparing hospital performance might do well to account for such factors in their assessment. If up to 70 percent of health outcomes are driven by factors beyond medical care, and measures used to compare hospitals focus only on clinical factors, are hospital rating systems getting it right?
Against a backdrop of evidence showing that social factors strongly influence many health outcomes, CMS is poised to launch a revamp of its star rating system, including Hospital Compare. Hospital Compare uses a 5-star rating system, with 5 stars as the highest rating.
Adjustment for SRFs may drive the next generation of rating systems. In addition to CMS’s Star Ratings, patients and payers have many competing rating systems to consider, including U.S. News & World Report, Healthgrades, IBM Watson Health Top 100 Hospitals, and Leapfrog, among others.
A recent article in NEJM Catalyst compared and ranked several of those hospital rating systems. The authors noted that different methods can produce conflicting results. The same hospital can rate highly in one ranking and poorly in another. The addition of risk adjustment for SRFs may result in greater consistency and less confusion across rating systems.
Neighborhood disadvantage affects the Medicare Hospital Compare program
A recent article in Medical Care highlights an evaluation of Hospital Compare. It suggests that hospital ratings would shift if adjustment for social factors were included. Fahrenbach and colleagues examined the relationship between neighborhood SRFs and hospital ratings in the Medicare Hospital Compare program using 2017 Hospital Compare ratings. The study looked at 3,608 hospitals across all 50 states. The outcome measures included a summary score (used to create CMS’s star ratings) and 7 quality group scores: care effectiveness, efficiency, hospital readmissions, in-hospital mortality, patient-reported care experiences, safety, and timeliness.
Methodological quirks
The authors used an unusual method for identifying hospital service areas. They did not use the Dartmouth Atlas’s Health Service Areas (which they argued are too broad and heterogenous) or actual inpatient utilization data to determine market area. Instead, they drew boundaries around each hospital based on the number of beds and a multiplier of 2.4 hospital beds per 1,000 people. Details are available in the supplementary digital content [Word].
This may have mis-estimated the service areas for some hospitals, including the authors’ own University of Chicago hospital. As an academic medical center, that hospital would be likely to draw patients from across the Chicago metropolitan area. It is not as dependent on South Side patients as the authors’ map would indicate.
Complex results
Methodological quibbles aside, the authors’ findings were complex. They reported some relatively large effects of SRFs on Hospital Compare measures. And some of the SRFs they examined had opposite effects on the same measure. The effects were measured on a percentage scale of 0-100, with 100 representing better performance after accounting for SRFs. For example:
- The mortality score:
- decreased by 1.6 points for every 10% increase in Medicare/Medicaid dual enrollment in the neighborhood; and
- increased by 3.4 points for every 10% increase in the percent of people with commutes longer than 45 minutes.
- The timeliness of care measure:
- decreased by 2.6 points for every 10% increase in the percent of people speaking a language other than English; and
- increased by 1.4 points for every 10% increase in unemployment.
The authors reported that SRFs had the biggest impact on timeliness of care and the smallest impact on the safety of care, with mortality in the middle. Overall, a given hospital’s summary score tended to decline as social risk increased.
The most relevant social risk factors
Many SRFs were associated with lower hospital ratings. These included:
- neighborhood-level percent of dual-eligible residents
- lower median home values
- percent with a high school diploma
- percent unemployed
- percent of black residents
- percent speaking a language other than English
- percent with longer commute times
Factors not associated with ratings included neighborhood measures of family income, percent of single-parent households, percent uninsured, and household size. A sensitivity analysis used the Area Deprivation Index, a composite SRF measure that has been correlated with a number of health outcomes. The authors found that a 10-point increase in area deprivation was associated with a 1.2-point reduction in the overall Hospital Compare summary measure.
Going the next step, the authors looked at the effects of adjusting for SRFs in computing hospital star ratings. They found that for two-thirds of hospitals, adjusting would make no difference. But for the remaining hospitals, their ratings would change by one star. Generally, the rank order of ratings did not change. Instead, the range of performance across hospitals became narrower.
Implementing adjustment for SRFs
Some critics have raised the concern that adjusting for SRFs could hide disparities by masking lower-quality care provided to disadvantaged populations. Others argue that it is not fair or equitable to hold providers accountable for factors outside their control. Experts from the National Quality Forum, the Department of Health and Human Services, and the National Academies (among others) have all endorsed the idea that performance measurement should consider SRFs. The National Academies report issued in 2017 specifically stated that:
The trend toward [value-based payment] could result in certain adverse consequences for socially at-risk populations, such as leading providers and health plans to avoid patients with social risk factors, underpayment of providers disproportionately serving socially at-risk populations (e.g., safety-net providers), and thus exacerbating health disparities.
Since 2017, CMS has adjusted some publicly reported quality measures for dual enrollment (as a marker of poverty) and disability. In Massachusetts’s Medicaid program, plan reimbursements are adjusted for housing instability, behavioral health issues, disability, and neighborhood-level stressors. Minnesota’s integrated health partnerships have also applied payment adjustment for social risk. Other state Medicaid programs are considering doing so in the near future.
More research is needed on which SRFs are most important and how these factors interact. Meanwhile, the research by Fahrenbach and colleagues provides useful information on how hospital performance measures could be affected by area-level disadvantage.
Lisa M. Lines
Latest posts by Lisa M. Lines (see all)
Kara Sokol