Speaker: Kristen Olson (Nebraska)
Title: "The effect of question and questionnaire characteristics on interviewer and respondent behaviors in CATI surveys."
Abstract:
In this paper, we evaluate the joint effects of question, respondent and interviewer characteristics on two proxy indicators of data quality - response time and question misreading - in a telephone survey. We include question features traditionally examined, such as the length of the question and format of response options, and features that are related to the layout and format of interviewer-administered questions. First, we examine how these question features affect the time to ask and answer survey questions and how different interviewers vary in their administration of these questions. Second, we investigate how choices in visual design features in particular, that is design features that require interviewer decisions, contribute to interviewer question misreading. These two measures of question time and question misreading are both proxies for the risk of measurement error in responses to survey questions.
To examine these questions, we use paradata and behavior codes from the Work and Leisure Today (n=450, AAPOR RR3=6.3%) survey and use cross-classified random effects models. Overall, more of the variation in both response time and question misreading is due to question characteristics compared to respondent or interviewer attributes. Additionally, we find that question characteristics related to necessary survey design features and respondent confusion are the primary predictors of response time, with little effect of visual design features of the question. Our results for question misreading show a different pattern. Characteristics related to task complexity and visual design significantly affect question misreading, with little contribution of necessary survey design features. We conclude with implications for survey practice.(archived presentation)
Location:
- Carnegie Mellon: contact William Eddy (bill@cmu.edu)
- Census Bureau headquarters: Room T5, contact Nancy Bates (nancy.a.bates@census.gov)
- Cornell University, Ithaca campus: Ives 105, contact Lars Vilhuber (lars.vilhuber@cornell.edu)
- Duke University: contact Jerry Reiter (jerry@stat.duke.edu)
- University of Michigan: Room 3443 ISR-Thompson, contact Maggie Levenstein (maggiel@umich.edu)
- University of Missouri: contact Scott Holan (holans@missouri.edu)
- University of Nebraska-Lincoln: Room TBD: contact: Kristen Olson (kolson5@unl.edu)
- Northwestern University: contact Zach Seeskin (z-seeskin@u.northwestern.edu)
- Streaming video: [click here] (link active about 5 minutes after start of seminar)
