The Effects of Using Likert vs. Visual Analogue Scale Response Options on the Outcome of a Web-based Survey of 4th Through 12th Grade Students

Abstract

For more than a half century surveys and questionnaires with Likert-scaled items have been used extensively by researchers in schools to draw inferences about students; however, to date there has not been a single study that has examined whether alternative item response types on a survey might lead to different results than those obtained with Likert scales in a K-12 setting. This lack of direct comparisons leaves the best method of framing response options in educational survey research unclear. In this study, 4th through 12th grade public school students were administered two versions of the same survey online: one with Likert-scaled response options and the other with visual analogue-scaled response options. A randomized, fixed-effect, between-subjects experimental design was implemented to investigate whether the survey with visual analogue-scaled items yielded results comparable to the survey with Likert-scaled items based on the following four methods and indices: 1) factor structure; 2) internal consistency and test-retest reliability; 3) survey summated scores; and 4) main, interaction, and simple effects. Results of the first three indices suggested that both the Likert scale and visual analogue scale produced similar factor structures, were equally reliable, and yielded summated scores that were not significantly different across all three school levels (elementary, middle, and high school). Results of the factorial ANOVA suggested that only the main effect of school level was statistically significant but that there was no significant interaction between item response type and school level. Results of the post-survey questionnaires suggested that students at all school levels preferred answering questions on the survey with the VAS compared to the LS nearly three to one.