TY - JOUR T1 - Do Interviewer Post-survey Evaluations of Respondents Measure Who Respondents Are or What They Do? A Behavior Coding Study JF - Public Opinion Quarterly Y1 - 2017 A1 - Kirchner, Antje A1 - Olson, Kristen A1 - Smyth, Jolene D. AB - Survey interviewers are often tasked with assessing the quality of respondents’ answers after completing a survey interview. These interviewer observations have been used to proxy for measurement error in interviewer-administered surveys. How interviewers formulate these evaluations and how well they proxy for measurement error has received little empirical attention. According to dual-process theories of impression formation, individuals form impressions about others based on the social categories of the observed person (e.g., sex, race) and individual behaviors observed during an interaction. Although initial impressions start with heuristic, rule-of-thumb evaluations, systematic processing is characterized by extensive incorporation of available evidence. In a survey context, if interviewers default to heuristic information processing when evaluating respondent engagement, then we expect their evaluations to be primarily based on respondent characteristics and stereotypes associated with those characteristics. Under systematic processing, on the other hand, interviewers process and evaluate respondents based on observable respondent behaviors occurring during the question-answering process. We use the Work and Leisure Today Survey, including survey data and behavior codes, to examine proxy measures of heuristic and systematic processing by interviewers as predictors of interviewer postsurvey evaluations of respondents’ cooperativeness, interest, friendliness, and talkativeness. Our results indicate that CATI interviewers base their evaluations on actual behaviors during an interview (i.e., systematic processing) rather than perceived characteristics of the respondent or the interviewer (i.e., heuristic processing). These results are reassuring for the many surveys that collect interviewer observations as proxies for data quality. UR - https://doi.org/10.1093/poq/nfx026 ER - TY - JOUR T1 - Do Interviewers with High Cooperation Rates Behave Differently? Interviewer Cooperation Rates and Interview Behaviors JF - Survey Practice Y1 - 2016 A1 - Olson, Kristen A1 - Kirchner, Antje A1 - Smyth, Jolene D. AB - Interviewers are required to be flexible in responding to respondent concerns during recruitment, but standardized during administration of the questionnaire. These skill sets may be at odds. Recent research has shown a U-shaped relationship between interviewer cooperation rates and interviewer variance: the least and the most successful interviewers during recruitment have the largest interviewer variance components. Little is known about why this association occurs. We posit four hypotheses for this association: 1) interviewers with higher cooperation rates more conscientious interviewers altogether, 2) interviewers with higher cooperation rates continue to use rapport behaviors from the cooperation request throughout an interview, 3) interviewers with higher cooperation rates display more confidence which translates into different interview behavior, and 4) interviewers with higher cooperation rates continue their flexible interviewing style throughout the interview and deviate more from standardized interviewing. We use behavior codes from the Work and Leisure Today Survey (n=450, AAPOR RR3=6.3%) to evaluate interviewer behavior. Our results largely support the confidence hypothesis. Interviewers with higher cooperation rates do not show evidence of being “better” interviewers. VL - 9 UR - http://www.surveypractice.org/index.php/SurveyPractice/article/view/351 IS - 2 ER - TY - ABST T1 - Are Self-Description Scales Better than Agree/Disagree Scales in Mail and Telephone Surveys? Y1 - 0 A1 - Timbrook, Jerry A1 - Smyth, Jolene D. A1 - Olson, Kristen ER - TY - ABST T1 - Are Self-Description Scales Better than Agree/Disagree Scales in Mail and Telephone Surveys? Y1 - 0 A1 - Timbrook, Jerry A1 - Smyth, Jolene D. A1 - Olson, Kristen ER - TY - ABST T1 - "During the LAST YEAR, Did You...": The Effect of Emphasis in CATI Survey Questions on Data Quality Y1 - 0 A1 - Olson, Kristen A1 - Smyth, Jolene D. ER - TY - ABST T1 - "During the LAST YEAR, Did You...": The Effect of Emphasis in CATI Survey Questions on Data Quality Y1 - 0 A1 - Olson, Kristen A1 - Smyth, Jolene D. ER - TY - ABST T1 - How do Mismatches Affect Interviewer/Respondent Interactions in the Question/Answer Process? Y1 - 0 A1 - Smyth, Jolene D. A1 - Olson, Kristen ER -