Development of a Rasch/Guttman Scenario Instrument to Measure Teachers' Use of Data to Inform Classroom Instruction

Abstract

Teachers in the United States are increasingly tasked with using data to inform their classroom instruction both through federal policies, such as the Every Student Succeeds Act (ESSA, 2016), and state policies requiring the use of teacher-determined data-driven goals for performance evaluations (MA 603 CMR 35.07). Many teachers, however, report that they feel underprepared to engage in this type of work (Dunn et al., 2013), also called Data-Driven Decision Making (DDDM). In addition, there is currently a limited set of instruments to measure the construct of using data to inform classroom instruction and the instruments that currently exist measure this construct using a typical Classical Test Theory design.This work developed an instrument called the Using Data to Inform classroom Instruction (UDII) scale to measure teachers’ use of data to inform classroom instruction. It used the Rasch/Guttman Scenario (RGS) methodology, an approach that develops scenarios that reflect the rich lived experiences of individuals (Antipkina & Ludlow, 2020; Ludlow et al., 2014). The RGS approach utilizes the Rasch model, part of the family of Item Response Theory models, which conceptualizes a construct as a hierarchical continuum. Scenario items and people are plotted on the same variable map, which allows for the development of rich descriptions of individuals at particular raw score locations on the continuum. An interpretative variable map is included to help schools and districts use the results of the survey. This work adds to the growing body of literature utilizing the RGS approach, as well as the literature focused on the use of data to inform classroom instruction (or DDDM). The UDII scale can be utilized by schools and districts who are engaged in the work of using data to inform classroom instruction to identify the current skillsets of teachers and/or teams of teachers to provide differentiated support, or it can be used before and after an intervention focused on using data to inform classroom instruction to measure change.