Estimating achievement gaps from test scores reported in ordinal “proficiency” categories

AD Ho, SF Reardon - Journal of Educational and Behavioral …, 2012 - journals.sagepub.com
Journal of Educational and Behavioral Statistics, 2012journals.sagepub.com
Test scores are commonly reported in a small number of ordered categories. Examples of
such reporting include state accountability testing, Advanced Placement tests, and English
proficiency tests. This article introduces and evaluates methods for estimating achievement
gaps on a familiar standard-deviation-unit metric using data from these ordered categories
alone. These methods hold two practical advantages over alternative achievement gap
metrics. First, they require only categorical proficiency data, which are often available where …
Test scores are commonly reported in a small number of ordered categories. Examples of such reporting include state accountability testing, Advanced Placement tests, and English proficiency tests. This article introduces and evaluates methods for estimating achievement gaps on a familiar standard-deviation-unit metric using data from these ordered categories alone. These methods hold two practical advantages over alternative achievement gap metrics. First, they require only categorical proficiency data, which are often available where means and standard deviations are not. Second, they result in gap estimates that are invariant to score scale transformations, providing a stronger basis for achievement gap comparisons over time and across jurisdictions. The authors find three candidate estimation methods that recover full-distribution gap estimates well when only censored data are available.
Sage Journals