Your dashboard shows sign-ups falling away on the plan selection screen, and everyone on the team has a different theory. One person blames the pricing table, another blames the form, and a third wants to redesign the whole page. Nobody has yet watched a person try to pick a plan.
UX analysis settles that debate with evidence. You collect what participants do and say, measure where they fail, code why they fail, and rank each problem by what it costs. Here is the process step by step, with the metrics, the formulas, and the mistakes that weaken a finding.
TL;DR
- UX analysis turns sessions, metrics, and feedback into a ranked list of problems your team can fix.
- Pair numbers that show where people fail with sessions that show why they fail.
- Plan on about five participants per qualitative round as a starting point and about 40 when you need a metric you can benchmark.
- Code sessions into themes, confirm each theme with a second source, then rate its severity.
- Report each finding with its evidence and a recommendation, then retest the fix with new participants.
What UX Analysis Is and What It Answers
UX analysis is the work of examining how people use a product, through recorded sessions, task metrics, survey answers, and customer feedback, to find where the experience breaks down and decide what to change. Product analytics tells you what happened on a page. Watching participants and coding what they say tells you why, which is the part a team needs before it commits design and engineering time.
A useful analysis answers three questions about one product decision:
- Where do participants fail, hesitate, or slow down?
- What causes each of those problems?
- Which problem should the team fix first?
UX research is the wider discipline that plans the study and collects the evidence. UX analysis is the stage that makes sense of that evidence, so it can only be as sharp as the questions the study set out to answer.
Know the Two Kinds of UX Data You Will Analyze
Most UX data is either quantitative (counts and measurements) or qualitative (observations and words). A second split matters just as much when you analyze it: behavioral data records what people do, and attitudinal data records what they say. The guide to quantitative and qualitative testing covers when to run each kind of study.
| Data type | Behavioral (what people do) | Attitudinal (what people say) |
| Quantitative | Task success rate, time on task, error rate, click paths, funnel drop-off | System Usability Scale (SUS) scores, rating scales, closed survey questions, NPS, CSAT |
| Qualitative | Session recordings, hesitation, backtracking, think-aloud observation | Interview answers, open survey responses, support tickets, reviews |
Keep the four groups distinct in your analysis. A participant who rates checkout as easy and then fails to finish it has given you two kinds of evidence, and the gap between them is a finding worth reporting.
NPS (Net Promoter Score) and CSAT (Customer Satisfaction) measure loyalty and satisfaction. They add useful context, but they are not usability metrics, so a strong NPS says little about whether a specific flow is easy to complete.
Collect UX Data From Sources That Fit Your Question
Each source answers a different part of the question, and each has a blind spot. Choose the sources that cover the decision you need to make, then plan to cross-check them.
| Source | What it shows you | Where it falls short |
| Moderated usability sessions | Why participants hesitate, with live follow-up questions | Smaller samples and more researcher time per session |
| Unmoderated usability sessions | Recorded behavior and task success from more participants, faster | No live probing unless the study adds follow-up questions |
| Product analytics | Where people drop off, at the scale of your full traffic | Cannot explain the cause of a drop |
| Surveys | Ratings and attitudes from many respondents | Self-reported, so answers reflect memory and opinion |
| Support tickets and reviews | Problems customers raise without being asked | Skews toward people with strong feelings |
Moderated testing suits flows where you expect to ask “what were you looking for there?” in the moment. Unmoderated studies suit questions where you need more task completions quickly, and the tasks can stand on their own.
Size the study to the kind of answer you need. In a Nielsen Norman Group article, Jakob Nielsen argues that a qualitative round needs no more than five participants, and that a budget for 15 people is better spent on three rounds of five. For quantitative studies, Nielsen Norman Group recommends 40 participants in most cases, since smaller samples produce numbers too unstable to benchmark.
Keep the study script identical across sessions: the same tasks, wording, and instructions for every participant.
How to Do a UX Analysis in Eight Steps
The steps below take you from raw sessions to a ranked list of fixes. Steps 3 and 4 can run in parallel when one study produces both metrics and recordings.
1. Write Down the Decision the Analysis Has to Inform
Start with the product decision, then the research questions that feed it. “Can first-time buyers choose the right plan without contacting sales?” gives the analysis a target; “how is the pricing page doing?” does not.
Name the participant group and the journey stage too. If your team works with personas, tag every session with the persona it represents, so you can see whether a problem affects one group or everyone.
Map the steps of the journey before you open the data. A finding tied to a specific step of a specific journey is far easier for a stakeholder to place than a general complaint about “the site.”
2. Organize and Clean the Raw Data
Messy data slows every later step, so spend time here before you start coding.
- Store everything together. Put sessions, transcripts, survey exports, and metric sheets in one place, labeled by study, date, task, and participant ID.
- Remove incomplete records. Set aside a session that ended before the first task or a survey abandoned halfway, and note how many you removed and why. Try to distinguish technical failures or invalid sessions caused by usability problems but retain usable data from partially completed sessions.
- Flag outliers before you drop them. A participant who took ten minutes on a two-minute task may have hit the most serious problem in the study, so watch the session first.
- Check the study as well as the design. If several participants misread the same instruction, the problem sits in the script, and those task results need a caveat.
3. Calculate the Core Usability Metrics
Four metrics cover most usability studies. The formulas below are the ones to use in your spreadsheet, and the five most important usability metrics explain each one in more depth.
| Metric | How to calculate it | What to watch for |
| Task success rate | Participants who completed the task, divided by participants who attempted it, multiplied by 100 | Decide before the study how you will score partial success, and report it separately |
| Time on task | Average time from task start to task completion, using successful attempts | A few slow sessions pull the average up, so look at the spread as well; think-aloud sessions run slower than silent ones |
| Error rate | Errors observed, divided by the number of error opportunities, multiplied by 100 | Separate slips (accidental) from mistakes (misunderstandings), since they need different fixes |
| System Usability Scale (SUS) | Ten-item questionnaire answered on a 1 to 5 scale. Convert each answer first (odd-numbered items: answer minus 1; even-numbered items: 5 minus answer), add the ten results for a total of 0 to 40, then multiply by 2.5 for a score of 0 to 100 | A SUS score is not a percentage; above 68 is generally considered above average |
The System Usability Scale is most useful for comparing versions of a design over time. A single score on its own says little until you have a previous round or a benchmark next to it.
Report every rate with its sample. “7 of 8 participants completed checkout, or 88%” tells a stakeholder how much weight the number can carry; “88% success” on its own hides that.
4. Code Qualitative Data Into Themes
Nielsen Norman Group describes thematic analysis as tagging observations and quotes with codes so that patterns across participants become visible. Work through it in this order:
- Read or watch everything once before you code, including sessions you moderated yourself.
- Tag each observation or quote with a short code that says what it is about, such as “unclear plan difference” or “looking for price per seat.”
- Group related codes into candidate themes, and give each theme a one-line description.
- Take a break, then check each theme against the raw data. Merge or drop any theme that rests on a single participant or one memorable quote.
Count how many participants each theme appears in. “6 of 8 participants scrolled past the plan comparison without opening it” is a finding; “some people found pricing confusing” is an impression.
When several people code together, an affinity diagram on a shared board helps the team agree on themes quickly. Write the research questions at the top of the board so the grouping stays tied to the decision.
5. Cross-check Findings with Other Sources
A theme gets stronger when a second, independent source points the same way. Nielsen Norman Group calls this triangulation: combining methods so the limits of one are offset by another.
If analytics shows the drop on the plan screen and six of eight participants hesitate there for the same reason, you have both the where and the why. A single session can reveal something important and the harder job is spotting what repeats across the study.
State the limit inside the finding. Eight participants can show that a problem exists and why it happens; they cannot tell you what share of all your customers will hit it.
6. Rate Each Finding by Severity
A long list of findings stalls a team, so rank it. Jakob Nielsen’s severity ratings for usability problems weigh how often a problem occurs, how hard it is to overcome, and whether it keeps recurring, then combine them into one score.
| Rating | Meaning | What to do with it |
| 0 | Not a usability problem | Remove it from the list |
| 1 | Cosmetic problem | Fix only if time allows |
| 2 | Minor usability problem | Low priority |
| 3 | Major usability problem | High priority |
| 4 | Usability catastrophe | Fix before release |
Have two or three people rate each finding on their own, then discuss where they differ. Next to each rating, note the business context, such as the conversion step it affects or the support tickets tied to it, so decision-makers can see the cost of leaving it unfixed.
7. Report Findings Stakeholders Can Use
Give every finding the same four parts, so a reader can scan the report and compare findings side by side:
- Finding: what happened, with the count (“6 of 8 participants skipped the plan comparison”).
- Evidence: a short clip or quote, plus the metric that supports it.
- Severity: the 0 to 4 rating and the journey step it affects.
- Recommendation: the change you propose to test next.
Lead with the three to five highest-severity findings and move the rest to an appendix. A short clip of a participant searching for the plan comparison can make the case in a readout faster than a slide of percentages. Our guide to sharing research insights covers readouts, highlight reels, and transcripts in more depth.
8. Fix, Retest and Compare
Ship the change for the top finding, then run the same tasks with new participants. Compare task success, time on task, and the theme counts with the first round, and close the finding once the problem is gone and nothing new has appeared on that step.
Running these rounds on a regular rhythm, a practice known as continuous discovery, keeps the analysis current as the product changes.
Need help running the analysis? Userlytics’ UX Consulting team can plan the study, moderate the sessions and deliver the report with recommendations. Talk to a UX consultant
Where AI Fits in UX Analysis
AI now handles much of the first pass on qualitative data: transcription, tagging, clip selection, and first-draft summaries. The researcher still sets the questions, checks the evidence, and decides what the findings mean.
On Userlytics, these features cover different parts of the eight steps:
- AI Annotations (step 4): flags key moments, sentiment patterns, and usability friction once a transcript is ready, and packages them into clips and highlight reels. Every AI annotation is labeled, and you can edit, delete, or add to it.
- AI Insights (step 4): summarizes the sessions and answers plain-language questions about the study, citing the sessions, timestamps, and annotations behind each answer.
- Sentiment Analysis (step 4): marks sections of a transcript as positive, negative, or neutral in tone, and you can approve or change the result.
- Qual Reporting and Quant Reporting (steps 3, 4 and 7): turn a completed study into a shareable report, with themes, key findings, strategic areas and recommendations on the qualitative side and charts with explanations on the quantitative side.
For instance, a Userlytics customer had no dedicated UX researcher, so its VP of Marketing led a usability study with help from Userlytics’ UX Consulting team. The VP estimated that AI Insights cut analysis from 10 to 20 hours of manual work to one to two hours.
Userlytics’ AI UX tools allow you to trace back all the summaries and analysis back to the sessions. Open the cited clip before a theme goes into a report, and treat any summary as the start of steps 4 to 6 rather than a substitute for them.
Common UX Analysis Mistakes to Avoid
Most weak findings trace back to one of these habits. Each has a simple correction.
| Mistake | What it looks like | What to do instead |
| Reading only the memorable sessions | The loudest quote becomes the headline finding | Code every session before choosing quotes |
| Confirmation bias | Themes match what the team already believed, and contradictions go unmentioned | Have a second person code a sample of sessions and look for evidence against each theme |
| Treating a small sample as a share of all customers | “75% of users can’t find pricing,” based on eight sessions | Report counts (“6 of 8 participants”) and save percentages for samples sized for them |
| Reading SUS as a percentage | “Our SUS of 72 means the product is 72% usable” | Compare SUS scores across versions and against the average of 68 |
| Mixing attitudes with behavior | A high satisfaction score presented as proof that a flow works | Report what participants did next to what they said |
| Deleting outliers unwatched | The slowest session disappears from the data set | Watch it first, since it may show the most serious problem |
| Stopping at the report | Findings presented, nothing retested | Retest the top fixes with new participants |
Pick One Flow and Analyze It This Month
Choose the flow your analytics flags with the steepest drop. Write down the decision you need to make about it, then run five participants through the same tasks.
Code the sessions, rate each finding by severity, and fix the highest-rated problem before the next sprint ends. Then repeat the tasks with five new participants. If the theme disappears and task success rises, you have a result your stakeholders can see in the clips themselves.
