The Compass

Senior healthcare executives often inquire about benchmarking, seeking to compare their organizations with top-rated hospitals as defined by U.S. News & World Report. Typically, the goal is to learn from these high-performing peers and elevate their own performance. The problem, however, is that hospitals are comparing themselves against organizations that are fundamentally different with respect to the most important performance variable of all: market characteristics (e.g., competitive dynamics, population demographics, payer networks).

The role of benchmarking is to facilitate improvement

Benchmarking is defined as the process of measuring an organization’s performance against those of comparable organizations, with the goal of identifying internal opportunities for improvement.¹ Health economy executives must correctly identify their “true peers” since benchmarking against an aspirational peer is ineffective for performance improvement.

Benchmarking within healthcare has seen little evolution over time. While U.S. News & World Report has remained at the forefront of hospital ranking across the industry, the methodology has been heavily scrutinized in recent months.² Although the publication expects this year’s hospital rankings to reflect all previously participating hospitals and has announced weighting changes to its methodology, several measures included in the rankings have little relevance to the daily challenges of hospital operations. While consumers are the intended audience for the U.S. News & World Report rankings, the reality is that health economy executives benchmark their own performance based on these rankings.

The current benchmarking landscape lacks a holistic view

Historically, traditional hospital benchmarking has not equipped health economy stakeholders with the ability to identify relevant hospital peers. The existing benchmarking resources, which rely primarily on quality measures coupled with subjective criteria, have received criticism from both clinicians and academics in past years, with one group of researchers citing prevalent issues across lists, including limited data, lacking data audits and varying methods for compiling and weighting measures.³

Existing hospital rankings and ratings provide ordinal scores or ordered lists (i.e., best to worst) based in part on a variety of quality-centric measures, including HCAHPS, 30-day risk-adjusted mortality rate and readmission rates. Over time, the “best” or “top” hospital lists have become an element in strategic planning despite being designed for consumers. The U.S. News & World Report rankings purport to help consumers understand the “best” place to receive certain types of healthcare services, while Leapfrog Group, scores hospitals on patient safety (Figure 1).⁴ Healthgrades provides a review of clinical outcomes across multiple conditions to identify the hospitals with the “best” outcomes.⁵ While CMS Care Compare is intended to educate patients and provide consumer-curated scores, it also is used to incentivize performance, with Federal reimbursement levels (i.e., Medicare, Medicaid) subject to change based on a hospital’s rating score.⁶

The current ratings and rankings lack comparative elements, which leads hospitals, health systems and other stakeholders in the health economy to make arbitrary and incomplete parallels between a particular hospital and some of the nation’s “top” hospitals. Hospitals do not know how dissimilar they are to some of the nation’s top hospitals on different metrics, nor do they know which hospitals are most similar to them.

Moreover, the well-documented unaffordability of U.S. healthcare is a critical part of the longstanding “Triple Aim,” but none of the existing benchmarking methodologies account for measures related to cost of care in tandem with quality. Are there some hospitals that have comparable quality but starkly different prices? Absolutely. Insight into a broader set of measures across comparable organizations are critical components of improving performance in a way that moves the needle.

The future of benchmarking in healthcare

Over 4,000 U.S. hospitals rely on internal benchmarking through patient and physician experience surveys (i.e., Press Ganey), and yet those same hospital executives often ask me how to reconcile their generally positive survey results with unfavorable share of care (“wallet”), which underlies overall hospital financial performance.⁷

Benchmarking has the power facilitate healthy competition, promote transparency and shed light on the successes and challenges at the hospital level. It is a critical function of helping hospitals operate at the top of license and deliver high-value care. In a health economy grappling with financial and operational challenges, stakeholders cannot afford to rely solely on traditional "gold standard" sources like U.S. News & World Report. Rather, performance improvement strategies should stem from objective and more comprehensive evidence-based benchmarks.

Thanks to Katie Patton for her research support.