Exploring the Validity of Behavior Coding

Behavior coding is used to detect issues in questionnaires by flagging interactions between survey respondents and interviewers when their interactions deviate from standardized interviewing. It has become a popular method for evaluating survey questionnaire quality and has been used mostly in Western countries, with little research being done in Asia. However, because of its dependence on overt behaviors observed during the coding, it was hypothesized that it might not be as effective in a country where different communicative norms are present. We conducted an experiment in Korea using two sets of survey questionnaires with embedded problematic questions in the experimental group to investigate the validity of behavior coding as reference to the control group. Using 200 audio recordings from a telephone survey, we investigated whether behavior-coded data were helpful in identifying embedded flaws. The results indicated behavior coding in Korean surveys may not be as effective as in Western countries.