TY - JOUR T1 - Do Interviewer Post-survey Evaluations of Respondents Measure Who Respondents Are or What They Do? A Behavior Coding Study JF - Public Opinion Quarterly Y1 - 2017 A1 - Kirchner, Antje A1 - Olson, Kristen A1 - Smyth, Jolene D. AB - Survey interviewers are often tasked with assessing the quality of respondents’ answers after completing a survey interview. These interviewer observations have been used to proxy for measurement error in interviewer-administered surveys. How interviewers formulate these evaluations and how well they proxy for measurement error has received little empirical attention. According to dual-process theories of impression formation, individuals form impressions about others based on the social categories of the observed person (e.g., sex, race) and individual behaviors observed during an interaction. Although initial impressions start with heuristic, rule-of-thumb evaluations, systematic processing is characterized by extensive incorporation of available evidence. In a survey context, if interviewers default to heuristic information processing when evaluating respondent engagement, then we expect their evaluations to be primarily based on respondent characteristics and stereotypes associated with those characteristics. Under systematic processing, on the other hand, interviewers process and evaluate respondents based on observable respondent behaviors occurring during the question-answering process. We use the Work and Leisure Today Survey, including survey data and behavior codes, to examine proxy measures of heuristic and systematic processing by interviewers as predictors of interviewer postsurvey evaluations of respondents’ cooperativeness, interest, friendliness, and talkativeness. Our results indicate that CATI interviewers base their evaluations on actual behaviors during an interview (i.e., systematic processing) rather than perceived characteristics of the respondent or the interviewer (i.e., heuristic processing). These results are reassuring for the many surveys that collect interviewer observations as proxies for data quality. UR - https://doi.org/10.1093/poq/nfx026 ER - TY - JOUR T1 - Examining Changes of Interview Length over the Course of the Field Period JF - Journal of Survey Statistics and Methodology Y1 - 2017 A1 - Kirchner, Antje A1 - Olson, Kristen AB - It is well established that interviewers learn behaviors both during training and on the job. How this learning occurs has received surprisingly little empirical attention: Is it driven by the interviewer herself or by the respondents she interviews? There are two competing hypotheses about what happens during field data collection: (1) interviewers learn behaviors from their previous interviews, and thus change their behavior in reaction to the behaviors previously encountered; and (2) interviewers encounter different types of and, especially, less cooperative respondents (i.e., nonresponse propensity affecting the measurement error situation), leading to changes in interview behaviors over the course of the field period. We refer to these hypotheses as the experience and response propensity hypotheses, respectively. This paper examines the relationship between proxy indicators for the experience and response propensity hypotheses on interview length using data and paradata from two telephone surveys.Our results indicate that both interviewer-driven experience and respondent-driven response propensity are associated with the length of interview. While general interviewing experience is nonsignificant, within-study experience decreases interview length significantly, even when accounting for changes in sample composition. Interviewers with higher cooperation rates have significantly shorter interviews in study one; however, this effect is mediated by the number of words spoken by the interviewer. We find that older respondents and male respondents have longer interviews despite controlling for the number of words spoken, as do respondents who complete the survey at first contact. Not surprisingly, interviews are significantly longer the more words interviewers and respondents speak. VL - 5 SN - 2325-0984 UR - http://dx.doi.org/10.1093/jssam/smw031 IS - 1 ER - TY - JOUR T1 - Do Interviewers with High Cooperation Rates Behave Differently? Interviewer Cooperation Rates and Interview Behaviors JF - Survey Practice Y1 - 2016 A1 - Olson, Kristen A1 - Kirchner, Antje A1 - Smyth, Jolene D. AB - Interviewers are required to be flexible in responding to respondent concerns during recruitment, but standardized during administration of the questionnaire. These skill sets may be at odds. Recent research has shown a U-shaped relationship between interviewer cooperation rates and interviewer variance: the least and the most successful interviewers during recruitment have the largest interviewer variance components. Little is known about why this association occurs. We posit four hypotheses for this association: 1) interviewers with higher cooperation rates more conscientious interviewers altogether, 2) interviewers with higher cooperation rates continue to use rapport behaviors from the cooperation request throughout an interview, 3) interviewers with higher cooperation rates display more confidence which translates into different interview behavior, and 4) interviewers with higher cooperation rates continue their flexible interviewing style throughout the interview and deviate more from standardized interviewing. We use behavior codes from the Work and Leisure Today Survey (n=450, AAPOR RR3=6.3%) to evaluate interviewer behavior. Our results largely support the confidence hypothesis. Interviewers with higher cooperation rates do not show evidence of being “better” interviewers. VL - 9 UR - http://www.surveypractice.org/index.php/SurveyPractice/article/view/351 IS - 2 ER - TY - ABST T1 - Mismatches Y1 - 2016 A1 - Smyth, Jolene A1 - Olson, Kristen ER - TY - ABST T1 - Are Self-Description Scales Better than Agree/Disagree Scales in Mail and Telephone Surveys? Y1 - 0 A1 - Timbrook, Jerry A1 - Smyth, Jolene D. A1 - Olson, Kristen ER - TY - ABST T1 - Are Self-Description Scales Better than Agree/Disagree Scales in Mail and Telephone Surveys? Y1 - 0 A1 - Timbrook, Jerry A1 - Smyth, Jolene D. A1 - Olson, Kristen ER - TY - ABST T1 - "During the LAST YEAR, Did You...": The Effect of Emphasis in CATI Survey Questions on Data Quality Y1 - 0 A1 - Olson, Kristen A1 - Smyth, Jolene D. ER - TY - ABST T1 - "During the LAST YEAR, Did You...": The Effect of Emphasis in CATI Survey Questions on Data Quality Y1 - 0 A1 - Olson, Kristen A1 - Smyth, Jolene D. ER - TY - ABST T1 - The Effect of Question Characteristics, Respondents and Interviewers on Question Reading Time and Question Reading Behaviors in CATI Surveys Y1 - 0 A1 - Olson, Kristen A1 - Smyth, Jolene A1 - Kirchner, Antje ER - TY - ABST T1 - The Effect of Question Characteristics, Respondents and Interviewers on Question Reading Time and Question Reading Behaviors in CATI Surveys Y1 - 0 A1 - Olson, Kristen ER - TY - ABST T1 - The Effects of Respondent and Question Characteristics on Respondent Behaviors Y1 - 0 A1 - Ganshert, Amanda A1 - Olson, Kristen A1 - Smyth, Jolene ER - TY - ABST T1 - Going off Script: How Interviewer Behavior Affects Respondent Behaviors in Telephone Surveys Y1 - 0 A1 - Kirchner, Antje A1 - Olson, Kristen A1 - Smyth, Jolene ER - TY - ABST T1 - How do Low Versus High Response Scale Ranges Impact the Administration and Answering of Behavioral Frequency Questions in Telephone Surveys? Y1 - 0 A1 - Sarwar, Mazen A1 - Olson, Kristen A1 - Smyth, Jolene ER - TY - ABST T1 - How do Mismatches Affect Interviewer/Respondent Interactions in the Question/Answer Process? Y1 - 0 A1 - Smyth, Jolene D. A1 - Olson, Kristen ER - TY - ABST T1 - Interviewer Influence on Interviewer-Respondent Interaction During Battery Questions Y1 - 0 A1 - Cochran, Beth A1 - Olson, Kristen A1 - Smyth, Jolene ER - TY - ABST T1 - Response Scales: Effects on Data Quality for Interviewer Administered Surveys Y1 - 0 A1 - Sarwar, Mazen A1 - Olson, Kristen A1 - Smyth, Jolene ER - TY - ABST T1 - Why do Mobile Interviews Take Longer? A Behavior Coding Perspective Y1 - 0 A1 - Timbrook, Jerry A1 - Smyth, Jolene A1 - Olson, Kristen ER -