Writing effective questions for a discussion guide is critical in order to meet the research goals. When you start writing a discussion guide, it’s normal to worry about how many or how few questions you should prepare, whether or not questions are leading or confusing, and if they seem forced or unnatural.
The psychology is simple: view research participants as human. Prepare for the discussion with a great discussion guide, but also go into research conversations with the intent to actively listen and engage in meaningful dialogue.
In this blog post, we’ll dive into how to craft questions that pique contemplation and prompt narratives. Along the way keep in mind that discussions are just that, a conversation, and shouldn’t be a barrage of questions. The information you glean from asides, comments, and editorial flights of fancy could end up being just as valuable as the direct answers to explicit inquiries.
1. Work backward from what you want to learn.
If you don’t have clear and valid research goals to work backward from, it’s hard to write effective research questions. It’s important first to ensure the things you want to learn can in fact be answered with qualitative research. From there you can narrow the problem space.
Try gauging objective along one of these 4 simple problem-solving dimensions:
- Learn why, how, and when people do things: Qualifying the elements that influence behavior gives you much greater insight into the why behind people’s behavior. Suss out how and why people do certain activities in a user journey.
- How people feel about a process or experience: For example, this might mean asking research questions about pain points they have with product or service. Or you could pick a line of inquiry geared toward filing down the rough edges on a process.
- Measure people’s reaction to a new concept or idea: Incorporating user feedback early and often in the design process is key. You might ask questions to measure how participants feel about a new concept or idea in order to validate it.
- Understand the status quo and current workarounds: It’s helpful to know what we’re really up against before trying to create a new or novel solution. Ask questions to figure out how they solve a problem today or what their current method lacks can offer much-needed market positioning clarity.
2. Structure guide around research goals.
Every interview question you include in discussion guide should be directly related to research goals. Generally, effective discussion guide templates include the following sections:
- Introduction: Your discussion guide should introduce the dynamics – the guidelines, research team, research goals, and reasonable expectations – for the discussion, survey, or interview.
- Background: These questions are context-builders. They typically explore broader classifying demographic subjects for target audience – like age, identity, nationality, and employment status.
- Stimuli: This section is optional, and when used, it explores how the participant relates to the topic at hand. You might present images or other subject-introducing media here.
- Exploration: Exploratory questions are the direct part where you find out what the user feels or thinks about what you’ve presented.
- Conclusion: Leave space at the end for the participant to expand or elaborate on topics covered during the conversation. “Do you have anything else to add based on what we’ve talked about today?” Additionally, wrap-up by clearly communicating next steps (if any.)
3. Write separate discussion guides for different personas.
Your attention to detail should shine through in each participant’s interview experience. It’s important not to combine multiple user experience research questions that ought to remain separate. For example, if you’re a marketplace and want to understand a journey from different vantage points, it makes sense to create a separate discussion guide for each persona.
4. Help participants open up by building trust.
Many relevant personal questions, such as those regarding income, voting habits, and purchasing practices, can be sensitive. Try adding information that helps participants feel more inclined to answer genuinely as opposed to presenting the “best version” of themselves. I like to start my discussions off with icebreaker questions to shake off the nerves and encourage participants to trust me as the researcher and open up.
5. Be aware of leading questions and the “yes bias.”
Acquiescence bias, also known as “yes bias” or “agreeance bias” appears in the audience as a form of recurring false positives. In short, people agree with statements prompted by the researcher even if it isn’t a true reflection of how they feel.
Many factors can contribute to this form of bias and it can be tricky to solve since it’s occurring in the audience. Use initial questions to establish a behavioral baseline, then iteratively rewrite them to see if you’re unknowingly leading or influencing participants.
6. Prompt dialogue with open-ended questions.
Closed questions can be answered with yes or no, and are a quick way to stifle discussion. Open-ended questions on the other hand encourage the participant to engage in dialogue and use storytelling. It requires them to reply with hard facts. Open-ended questions are what will create rich, qualitative data. Open-ended research questions start with “describe why/how..” or “tell me more…” or “explain how/why…” or “walk me through…”
7. Ask past-based questions instead of future-based.
Beware of asking participants to predict the future or speculate on their future behavior. This is called a future-based question and it has no place in a qualitative research setting. It isn’t reliable because there is typically a delta between what people want and say they will do with what they actually will do. An example of a good past-based question might sound like “Think about the last time you ordered takeout. Walk me through the experience of placing order.”
8. Choose neutral wording.
Try policing tone to sidestep language that might nudge opinions one way or the other. This might be a good place to self-check for biased language, but nothing beats taking proactive steps. An example of a question written with bias is: “Describe how [product/service] made experience ordering takeout easier.” This question assumes the participant had a positive experience. Instead, write it with a neutral tone like “Describe experience ordering takeout with [product/service.]”
9. Write one “single-barreled” question at a time.
It’s tempting to want to cram as much as you can into a discussion guide, leading to a common mistake of including double-barreled questions. Also called compound questions in legal proceedings, double-barreled questions usually contain a conjunction like “and”. They can result in skewed or missing data because it asks the participant to respond to two topics or issues at the same time but only allow for answering one.
10. Solicit feedback on discussion guide.
Don’t trust yourself to identify biases – They wouldn’t be predispositions if people could somehow just turn them off with effortless grace. Talk to people outside peer, colleague, social, or other groups. Solicit their honest opinions on how questions come across.
Wrapping up
Creating effective user research questions for discussion guide takes time, diligence, and a willingness to refine. Fortunately, this is the perfect kind of work to tackle in pieces, gradually building up a knowledge library filled with trustworthy and rich qualitative data that will help teams make more informed and better decisions.