Understanding the CAHPS Star Ratings and Score Discrebency

The mismatch between CAHPS star ratings and score because the ratings are based on relative performance (within a group) and case-mix adjusted scores, not fixed thresholds or raw scores. 

 Why the Mismatch Happens

1. Relative Groupings Over Fixed Thresholds:

   • Star ratings are not assigned based on fixed thresholds. This means that the ratings depend on how a particular contract (e.g., health plan) performs compared to other contracts in a group.

   • Relative comparison is the key concept here. If a health plan performs well within its group but falls just short in absolute terms, it could still receive a higher star rating than expected.

2. Case-Mix Adjusted Scores:

   • Case-mix adjustment refers to the process of adjusting for differences in the types of patients or populations served. For instance, a health plan that serves sicker or more complex patients may have lower unadjusted scores, but after adjustment, its performance might be more favorable.

   • The adjusted scores are used for calculating the star ratings, not the raw survey results. This ensures a more accurate reflection of performance and accounts for differences in patient demographics or health status.

   This method of using adjusted rather than raw survey scores helps explain why the star ratings may not always align with the cutpoint thresholds. This allows for a more nuanced, fairer comparison across health plans.

For Reference: 2025 CAHPS Technical Specifications

Relative Distribution and Significance Testing (CAHPS) Methodology The CAHPS measures are case-mix adjusted to take into account differences in the characteristics of enrollees across contracts that may potentially impact survey responses. See Attachment A for the case-mix adjusters. The percentile cut points for base groups are defined by current-year distribution of case-mix adjusted contract means. Percentile cut points are rounded to the nearest integer on the 0-100 reporting scale, and each base group includes those contracts whose rounded mean score is at or above the lower limit and below the upper limit. The number of stars assigned is determined by the position of the contract mean score relative to percentile cutoffs from the distribution of contract weighted mean scores from all contracts (which determines the base group); statistical significance of the difference of the contract mean from the national mean along with the direction of the difference; the statistical reliability of the estimate (based on the ratio of sampling variation for each contract mean to between-contract variation); and the standard error of the mean score. All statistical tests, including comparisons involving standard errors, are computed using unrounded scores. CAHPS reliability calculation details are provided under the section header, “MA & PDP CAHPS Between Contract Variances for Reported Measures” at https://www.ma-pdpcahps.org/en/scoring-and-star-ratings. Tables K-9 and K-10 contain the rules applied to determine the final CAHPS measure star value.