Table of Contents
User research is as much an art as it is a science. What people say can be heavily biased by many factors—many of which have been scientifically proven in the fields of research, cognitive psychology and behavioral science. Bias in surveys and user interviews can distort results and confuse teams— leading to misinformed critical product decisions. Below are seven practical tips to help you reduce bias and improve the quality of your user research and survey questions.
Even experienced researchers can easily to fall into asking leading questions that push respondents toward a particular answer. For example, instead of asking, "How helpful did you find this feature?" you could ask, "How would you describe your experience with this feature?" Neutral wording helps reduce any preconceived notions or expectations. Any words that contain or imply subjective opinions can be heavily leading. Social desirability bias and acquiescence bias leads users to want to favorable or agreeable to the interviewer, rather than providing their true thoughts or behaviors.
Priming bias occurs when the order of your questions influences how respondents think or feel about later questions. Randomizing the order of questions for each participant can help mitigate this bias. For example, if you’re asking about both positive and negative experiences, leading with the positive can prime respondents to answer more positively in subsequent questions. This can also happen in multiple choice questions—where the first answer primes and anchors subsequent choices. So if you show 'very satisfied' as the first answer to a customer satisfaction question—users will be led to choose more positive answers. By shuffling questions or answers, you can prevent any single question from affecting responses to subsequent ones.
Asking users to predict their future behavior or preferences often leads to inaccurate answers. People are generally optimistic about the future and will answer favorably if you ask them 'Will you use this?' Think about all the new year resolutions that never get acted on! Instead, focus on gathering facts about their past behavior. For instance, instead of asking, "Would you use this feature in the future?" ask, "When was the last time you used a feature like this?" People tend to give more reliable responses when talking about actions they’ve already taken versus hypothetical scenarios.
What people say do not necessarily correlate with what they do. This is called the 'say-do gap' in user research. When possible, design your research to include tasks or actions that involve making an explicit personal choice, showing evidence or performing an action—rather than just asking for opinions. Exercises like asking people to sort or rank features/attributes (ie card sorting) or having them draw a graph that shows their emotional highs and lows across an experience—can be much more telling and aid in getting deeper answers. You can get pretty creative with this as well. I've heard of researchers asking participants to tear out images from magazines that represent their feelings about a brand to asking them to share their shopping cart history to discuss how they compared similar products.
Double-barreled questions ask about two things at once, which can confuse respondents and produce unreliable answers. For instance, “How satisfied are you with the interface and the customer service?” is asking about two distinct aspects of the product, which should be separated. Always ensure each question is focused on a single topic to avoid muddled responses.
Users’ experiences often vary based on their demographics, usage patterns, or familiarity with a product. By segmenting users based on their behavior (e.g., first-time vs. repeat users), you can tailor questions to capture insights that are more relevant to each group. This approach helps reduce generalizations and ensures that you gather more granular, actionable feedback.
Combining both qualitative and quantitative research methods can help reduce bias and cross-validate findings. For example, surveys provide numerical data that can reveal trends, while interviews or usability testing offer deeper context behind those numbers. By using a mix of methods, you can identify patterns, confirm hypotheses, and avoid relying too heavily on any single source of information.
The art of user research is in how the product manager, researcher or customer caring professional can best apply these techniques for their particular research goals and objectives. Even so, it's often that you will never have a completely clear picture of what users are saying or behaving to predict the types of product and business enhancements that will best satisfy their needs. Hopefully —armed with these types of bias reducing insights—you can improve the odds of your insights to be closer to what people actually want.
