Nominal and Effective Weights of Composite Accountability Ratings: A Demonstration Using Colorado’s School Performance Framework
Link to Resource: Nominal and Effective Weights of Composite Accountability Ratings: A Demonstration Using Colorado’s School Performance Framework
Authors: Benjamin R. Shear and Kaitlin Nath
Federal law requires states to create school accountability ratings based on multiple factors including student academic achievement, academic growth, and progress in achieving English language proficiency. Many states meet these requirements by creating weighted composite scores based on indicators for each factor. When a composite score is constructed as a weighted sum of multiple indicators, the weights are called nominal weights because they represent the intended weights being given to each indicator. The actual influence of each indicator on the overall composite is referred to as the effective weight of that indicator. When the nominal and effective weights of an indicator differ, it can undermine the validity of interpretations of the composite scores. We review a widely used method for calculating effective weights and apply it to evaluate the alignment of nominal and effective weights for composite accountability scores produced in the Colorado School Performance Framework (SPF). We find the alignment of the nominal and effective weights varies across grade levels, with higher alignment for the elementary and middle school scores than for high school scores. We discuss factors that influence this alignment, such as the correlation and relative variances among the included indicators, and implications for designing school accountability composite scores.