Using the Score-based Testlet Method to Handle Local Item Dependence

Abstract

Item Response Theory (IRT) is a contemporary measurement technique which has been used widely to model testing data and survey data. To apply IRT models, several assumptions have to be satisfied. Local item independence is a key assumption directly related to the estimation process. Many studies have been conducted to examine the impact of local item dependence (LID) on test statistics and parameter estimates in large-scale assessments. However, in the heath care field where IRT is experiencing greater popularity, few studies have been conducted to study LID specifically. LID in the health care field bears some unique characteristics which deserve separate analysis. In health care surveys, it is common to see several items that are phrased in a similar structure or items that have a hierarchical order of difficulties. Therefore, a Guttman scaling pattern, or a deterministic response pattern, is observed among those items. The purposes of this study are to detect whether the Guttman scaling pattern among a subset of items exhibit local dependence, whether such dependence has any impact on test statistics, and whether these effects differ when different IRT models are employed. The score-based approach - forming locally dependent dichotomous items into a polytomous testlet - is used to accommodate LID. Results from this dissertation suggest that the Guttman scaling pattern among a subset of items does exhibit moderate- to high-degree of LID. However, the impact of this special LID is minimal on internal reliability estimates and on the unidimensional data structure. Regardless of which models are employed, the dichotomously-scored person ability estimates are highly correlated with the polytomously-scored person ability estimates. However, the impact of this special LID on test information differs between Rasch models and non-Rasch models. Specifically, when only Rasch models are involved, test information derived from the LID-laden data is underestimated for non-extreme scores; whereas, when non-Rasch models are used, the opposite finding is reached –that is, LID tends to overestimate test information.