On the validity and sensitivity of the phonics screening check: erratum and further analysis

Copyright © 2017 UKLA Duff, Mengoni, Bailey and Snowling (Journal of Research in Reading, 38: 109–123; 2015) evaluated the sensitivity and specificity of the phonics screening check against two reference standards. This report aims to correct a minor data error in the original article and to present further analysis of the data. The methods used are calculation of predictive values of the phonics screening check in addition to sensitivity and specificity, and evaluation of agreement between the reference tests. Predictive values are important indicators of screening test quality. The positive predictive value of the phonics check is low (0.31) when compared with a standardised reading test but high (0.84) when compared with teachers' phonic phases judgements, reflecting poor agreement (kappa = 0.27) between reference tests. Results have implications for practice in terms of choice of reference standard and choice of threshold criterion for children to pass the screening check. Longitudinal data are needed to assess the predictive validity and utility of the check. What is already known about this topic: The importance of phonics in learning to read is widely acknowledged. The phonics screening check was introduced into U.K. schools in 2012 to ensure that all children develop phonic decoding skills. Estimates of the sensitivity and specificity of the phonics screening check, compared with two established ‘reference’ measures, were reported by Duff et al. ([Duff, F.J., 2015]). What this paper adds: We correct a minor error in the report of the original data by Duff et al. ([Duff, F.J., 2015]). We draw attention to the importance of including predictive values, alongside sensitivity and specificity, in the evaluation of screening test validity. We also propose an alternative statistic for comparing the two reference measures. We show that applying this further analysis to the data in Duff et al. ([Duff, F.J., 2015]) reveals the following: (i) the numbers of incorrect (false positive and false negative) outcomes in the phonics check and (ii) the marked difference in these numbers depending on the choice of reference measure. Implications for theory, policy or practice: Reports of screening test validity should include positive and negative predictive values. A fundamental consideration for evaluating the validity of the phonics screening check is the choice of reference measure. Longitudinal data are needed to assess the predictive validity and utility of the phonics check.