A test instrument calibration study: The Rasch Model approach to investigate Web-mediated instructional outcomes

Measuring cognitive performance is a complex process, which is made worse, by the number of mitigating variables. Consequently researchers must carefully calibrate their assessment instruments to ensure they test the intended learning outcomes. The purpose of this paper is to explain a methodology to determine test accuracy in an investigation of Web-mediated instructional outcomes. There is little evidence in the literature on human performance measurement when using educational ICT tools online. Previous research reveals there are interactive effects of paper-based instruction, within an individual¿s cognitive style construct. Whether these effects also relate to a Web-mediated context remains unclear. We describe a doctoral research study underway, which uses a quasi-experimental research design, involving: an exploratory study, the validation and reliability testing of the assessment instrumentation to prepare for the main experiment. The data were analysed using the Quest Interactive Test Analysis System that is based on the Rasch probabilistic measurement technique.