Synthesising results into valuable insights
Analysing UX research and synthesising the data that comes from it can play a huge part in the success of the end product. In this piece, we take a look at how analysing UX research will turn findings into useful insights.
UX research can be a valuable part of the UX design process. However, completing study is only the first step in obtaining the insights you seek. Until you analyse the data you’ve collected into a set of findings, synthesise those findings into insights and interpret the results, you won’t be able to communicate how research can help improve the user experience to stakeholders and clients.
In this post, we’ll discuss how analysing UX research using quantitative and qualitative data will turn findings into useful insights. We’ll also look at how to interpret findings. Here’s what we’ll cover:
What is user research analysis?
User research analysis involves analysing the data you’ve collected during study. Analysis can be done in a variety of ways. The kind of analysis you choose to perform will depend on whether you’ve collected qualitative or quantitative data and what you were hoping to learn from study. However, no matter what method you use, the goal of analysis is to identify the factual results of the study. In other words, it’s during analysis that you use the data you collected to arrive at a set of research findings.
What’s the difference between analysing UX research and synthesising UX research?
The findings you obtain from analysis may be interesting, but they don’t provide any real insight until you engage in synthesis. Synthesis is the process of bringing all the findings from analysis together to extract insights and conclusions from the data, as well as a set of actionable recommendations for the UX design of the product. While analysis provides a set of facts, synthesis makes those facts meaningful.
Analysis and synthesis often happen at the same time. Yet, while we plan our analysis in anticipation of the questions we want to answer, synthesis is an emergent process through which we make connections and come up with possible insights as we go. Throughout this post, we’ll talk about analysis and synthesis separately, but in reality, these processes are likely to overlap.
When should you conduct user research analysis?
The obvious answer to this question is after you’ve finished collecting data from user research study, but you’ll need to think about analysis even before you start user research. Before you begin, define a set of objectives and research questions that you want to answer and come up with ways to answer them.
Then, during the study, conduct analysis while data is being collected. Analysis can help you ensure you’re asking the right questions. For example, if you’re conducting user interviews and you conduct periodic analysis, you may find out you’re asking the wrong questions—ones that don’t really pertain to the variable you want to explore. The good thing is, if this is the case and you’ve caught it early on, you haven’t wasted the whole study on the wrong variable.
In addition to finding any errors, starting analysis during study is also more efficient. But the truth is the majority of the real work in user research analysis happens at the end of the study. No matter what, though, the important thing is that you finish analysing all the data you collected before you interpret the results and draw conclusions.
What are the different types of user research analysis? (Quantitative vs. qualitative)
There are two types of user research, quantitative and qualitative, and they each require a different type of analysis.
Quantitative research gathers objective, numerical data and can help you answer questions about things like success rates, task times, and error rates. Quantitative data tells you the “what” and the “how”—what do users do / how do they interact with the product?
Qualitative research gathers subjective, qualitative data, usually in the form of words (i.e. what the user says about their experience, how they feel, what they think about a product, etc.). When analysing qualitative research data, you’re looking for themes and patterns across users’ responses. Qualitative research can provide insight into what features are most important to users, how a given experience makes them feel, what they find difficult about a certain experience, and so on. Given the in-depth insights qualitative research can yield, analysing the results often takes longer than for quantitative research.
Now let’s take a closer look at how to analyse both quantitative and qualitative data.
How to analyse quantitative research data
If you’ve conducted a quantitative study, such as a survey with yes/no or multiple choice questions, an A/B test or an eye tracking test, you will be left with a large set of numerical data. Depending on how the data was collected, it will either already be laid out in a spreadsheet or will have to be entered into a spreadsheet manually, where each column corresponds to one question and each row includes one participant’s answers.
The dataset in the spreadsheet will then be analysed statistically. Programmes like R or SPSS can be used to run statistical analysis or formulas can be plugged into a Google or Excel spreadsheet.
Before you start running statistical formulas on data, go back to the original goals of the study and decide exactly what questions you want to answer.
For example, maybe you’re curious to learn how long it takes a user to sign up for a newsletter. Or perhaps you’re trying to find out if users are satisfied with the various steps of a checkout process. No matter what the goals are, make sure to concentrate analysis there.
The good thing about quantitative analysis is that once you decide on the variables you want to analyse, it can be extremely quick and efficient to perform. Quantitative findings, such as the average time to complete a task, participants’ satisfaction ratings with parts of a product or information on features they use the most, can lead to insights about whether the UX for a certain task should be refined and what features should be redesigned or eliminated from a product entirely.
Quantitative data can also be analysed to compare and contrast the way users from different demographic groups use a product. These findings can provide insight into the different use cases UX designers must keep in mind as they’re creating the product’s user experience.
How to analyse qualitative research data
If you’ve conducted a qualitative study, such as user interviews, focus groups or ethnographic research, you’ll be left with a large amount of information in the form of words. If participants didn’t provide their answers in written form, you’ll want to have all of the interviews or responses transcribed so you can easily read what participants said. While it can be expensive, it’s worthwhile to use a service like Rev.com to transcribe interviews so time is freed up to focus on other tasks.
Once the data is transcribed, you can organise it in a number of ways. One is to put it in a spreadsheet where each row represents the answers provided by a single participant. Another option is to upload that data to a qualitative analysis tool like NVivo or Dedoose.
Just like with quantitative data, before you settle on a method for analysing the data qualitatively, you should revisit the original goals of research and make sure that analysis focuses on them.
For example, if you’re designing a real estate app where users can find houses for sale, you’ll want to focus on the demographics of potential users, what features they focus on most when searching for a home and what draws their attention to a given listing.
Although participants might have brought up other topics during study, don’t include them in analysis if they don’t pertain to research goals.
There are several ways you can analyse qualitative data. Two popular options are content analysis and affinity mapping.
Content analysis
Content analysis involves looking for patterns in the data and then coding them. It can be especially useful for evaluating long text data such as interview transcripts. Codes are essentially labels that you can apply to each chunk of text that brings up a particular topic.
For example, for the real estate app mentioned above, you might use codes such as budget, location and number of bedrooms. As you go through the text data, you will then label each chunk of data where these subjects are discussed.
Here are the steps to perform a content analysis:
- Decide on codes. There are two ways to do this. On the one hand, you can come up with codes based on the topics you’re hoping to find. If you’re interested in seeing what people have to say on a house with a pool, you’ll include the code “pool” in the analysis. On the other hand, you can see what topics emerge organically. If you review the data and notice that decks come up frequently, you’ll include the code “deck” in analysis.
- Assign codes as you work. This can be done manually or with a qualitative analysis program like those mentioned above.
- Organise related codes into themes. Once you’ve coded all the data, look for codes that speak to the same general topic and place these under larger umbrella categories called themes. For example, codes like “gated community,” “good schools” and “safe location” can be grouped into a theme called “neighbourhood.”
If more than one member of the team is coding the data, you need to make sure everyone understands the codes the same way. To do this, before coding the entire dataset, each coder should code the same small part of the data and compare their work. If there is disagreement in the way the codes are applied, coders need to discuss the discrepancies until they’ve agreed on how to apply the codes consistently.
Affinity mapping
Another useful way to analyse the data is through affinity mapping. Affinity mapping is a visual way of organising the data but, like content analysis, the overall goal is to identify patterns and themes.
Although this is not the only way to conduct affinity mapping, User Interviews recommends taking these four steps:
- Write all the text data points on post-it notes. While this will likely lead to a lot of post-its, make sure to use as many as the project requires.
- Stick all the post-its to a wall, whiteboard, or any other large surface that people can gather around.
- Move the post-its into groups based on common themes that pertain to research objectives. You may want to limit this process to about 20-30 minutes depending on how many post-its you have to organise and to prevent the affinity mapping session from sprawling.
- Continue to re-organise the groups until time runs out or the team has come to a consensus. If time runs out and team members still disagree on the groups, have a discussion until everyone’s on the same page.
Once you complete this process, you may want to label each group with an overarching theme that sums up the content.
Synthesising and interpreting user research data and presenting findings
The final product of any user research analysis is interpreting research and presenting findings. presentation to stakeholders and clients should have sections for “key learnings” and “recommendations.” In the “key learnings” section you’ll interpret research data so it provides value. For example, say one of insights is that there was a theme of “budget” for the real estate app mentioned above. You interpret it in the form of a key learning, “Make sure to feature budget prominently and not share anything above this price range.”
Then the “recommendation” makes this insight valuable by framing it as a recommended action. For example, the “recommendations” for the above key learning could be “Make budget a major section of the website” and “do not share anything above the user’s price range, if they’ve shared their budget.” When possible, it helps to combine qualitative and quantitative findings for the biggest impact.
Interpreting the data is where you’ll really probe the analysis for meaningful insights, and it’s these insights that ultimately have the greatest benefit of any user research. Interpretation enables us to truly understand what users want, and don’t want, from the product we’re designing. Interpreting the research data and delivering a presentation that shares these insights increases our chances of creating product users will love.