Project InSe

Instructional Sensitivity of Test Items

Researchers as well as policy-makers regularly rely on student performance data to draw inferences on schools, teachers, or teaching. Yet, valid inferences drawn from student test scores require that instruments are sensitive to the instruction that students have received in class. Accordingly, measures of test and test items’ instructional sensitivity may provide empirical support for validity claims about the inferences on instruction derived from student test scores. The project InSe focuses on two questions:

• How can the instructional sensitivity of tests and items be measured empirically?
• How can information on instructional sensitivity be used in educational and psychological testing?

The project builds on a longitudinal multilevel DIF-model (LML-DIFmodel) for evaluating instructional sensitivity (Naumann, Hochweber, & Hartig, 2014). The LML-DIF model integrates current approaches for measuring the instructional sensitivity of items by considering both differences between time points of measurement as well as differences between learning groups (i.e. classes). Four major objectives are pursued:

• further development and extension of the LML-DIF model
• examination of methods and conditions of parameter estimation
• the validation of the statistical indicators of instructional sensitivity
• development of criteria to classify the instructional sensitivity on the test and the item level

Cooperation

University of Teacher Education St. Gallen (PHSG)

Selected Publications

Naumann, A., Hartig, J., & Hochweber, J. (2017). Absolute and relative measures of instructional sensitivity. Journal of Educational and Behavioral Statistics. Advance Online Publication. doi:10.3102//1076998617703649

Naumann, A., Hochweber, J., & Klieme, E. (2016). A psychometric framework for the evaluation of instructional sensitivity. Educational Assessment, 21(2), 89–101. doiI:10.1080/10627197.2016.1167591

Naumann, A., Hochweber, J., & Hartig, J. (2014). Modeling instructional sensitivity using a longitudinal multilevel differential item functioning approach. Journal of Educational Measurement, 51, 381–399. doi:10.1111/jedm.12051