Is your customer survey data accurate?

Customer and employee surveys are one of the best ways to obtain critical feedback and measure performance in key areas. However, the way survey questions are written and presented can have a measurable impact on the quality and accuracy of responses.

Formulating questions has become both a science and an art based on need, desired outcomes and extracting vital bits of information. In law enforcement, detectives and interrogators are trained on the what, when, and how of asking questions. Executive meeting facilitators are savvy in asking question upon question to drill deep into certain areas so they may emerge with golden nuggets of actionable information to help companies achieve key objectives. And, in the courtroom, attorneys are astute in knowing what questions to ask and how to frame them to help win a case. Some believe these practices to be manipulative while others believe them to be intelligent.

When drafting surveys, the purest form of feedback (free of any influence) can only be acquired when such tactics are not used to gather intelligence. However, without even knowing it, some survey authors employ a manipulative tactic referred to as a positive bias form of questioning. As the term implies, this form of questioning subconsciously influences survey respondents to lean toward the positive when answering certain questions, consequently producing skewed, inaccurate results.

Consider the case of a manufacturer of commercial dehumidifiers wishing to learn from contractors if its dehumidification units are easy to install. A seemingly obvious question to ask on a survey to contractors would be, “Are our dehumidification units easy to install and service?” with an option to select “yes” or “no” as a response. In this example, the positive bias comes into play by only using the word “easy” in the question. When thinking through how to answer the question, the contractor is ‘pre-suaded to think of the different ways the installation was easy to perform. In so doing, s/he will be more apt to select “yes” as his or her answer.

Conversely, if the survey question read, “Are our dehumidification units difficult to install?” we could apply the same logic and predict with a high level of confidence that the contractor would be directed to think of what problems s/he may have experienced during installation and select “yes” as an answer. However, most survey creators would never frame questions in this manner simply because the negative bias and pessimistic nature of the question is noticeable! (Of course, with your next survey, you could run a test between the two types of questions to validate these claims, but I would advise against it as you wouldn’t want to plant a seed of negative thoughts with any of your customers.)

For a balanced and unbiased survey, your best strategy is to frame qualitative questions on a scale. Continuing with the manufacturer’s objective of wanting to know if his company’s dehumidifiers are easy to install, the question would ideally read, “On a scale of 1-6, (1=very easy, 6=very difficult), please rate your experience with installing our dehumidification units.”

Take twenty or thirty minutes to review the last few surveys you completed and pay particular attention to how the questions were framed. If you notice any with a positive bias, chances are that answers are skewed toward the positive, which, as you now know are misleading to some degree.

Next time you prepare a survey, be more cognizant of this common mistake and structure questions that will eliminate a positive or negative bias.