Is your customer survey data accurate?

Customer and employee surveys are one of the best ways to obtain critical feedback and measure performance in key areas. However, the way survey questions are written and presented can have a measurable impact on the quality and accuracy of responses.

Formulating questions has become both a science and an art based on need, desired outcomes and extracting vital bits of information. In law enforcement, detectives and interrogators are trained on the what, when, and how of asking questions. Executive meeting facilitators are savvy in asking question upon question to drill deep into certain areas so they may emerge with golden nuggets of actionable information to help companies achieve key objectives. And, in the courtroom, attorneys are astute in knowing what questions to ask and how to frame them to help win a case. Some believe these practices to be manipulative while others believe them to be intelligent.

When drafting surveys, the purest form of feedback (free of any influence) can only be acquired when such tactics are not used to gather intelligence. However, without even knowing it, some survey authors employ a manipulative tactic referred to as a positive bias form of questioning. As the term implies, this form of questioning subconsciously influences survey respondents to lean toward the positive when answering certain questions, consequently producing skewed, inaccurate results.

Consider the case of a bank wishing to learn from its customers if their experiences with the bank’s loan officers was favorable. A seemingly obvious question to ask on a survey to customers would be, “Did you have a positive experience with your loan officer?” with an option to select “yes” or “no” as a response. In this example, the positive bias comes into play by only using the word “positive” in the question. When thinking through how to answer the survey question, customers would be ‘pre-suaded to think of the different ways their experiences were positive. In so doing, s/he will be more apt to select “yes” as his or her answer.

Conversely, if the survey question read, “Did you have a negative experience with your loan officer?” we could apply the same logic and predict with a high level of confidence that customers would be directed to think of what problems s/he may have experienced and select “yes” as an answer. However, most survey creators would never frame questions in this manner simply because the negative bias and pessimistic nature of the question is noticeable! (Of course, with your next survey, you could run a test between the two types of questions to validate these claims, but I would advise against it as you wouldn’t want to plant a seed of negative thoughts with any of your customers.)

For a balanced and unbiased survey, your best strategy is to frame qualitative questions on a scale. Continuing with the bank’s objective of wanting to know if its customers are having positive experiences with the bank’s loan officers, the question would ideally read, “On a scale of 1-6, (1=very favorable, 6=very unfavorable), please rate your experience with your loan officer.”

Take twenty or thirty minutes to review the last few surveys you completed and pay particular attention to how the questions were framed. If you notice any with a positive bias, chances are that answers are skewed toward the positive, which, as you now know are misleading to some degree.

Next time you prepare a survey, be more cognizant of this common mistake and structure questions that will eliminate a positive or negative bias.