Skip to content

Qualitative vs Quantitative UX Research: Pick the Evidence Your Next Decision Needs

 By Userlytics
 Published: Jul 02, 2024
 Updated: Nov 20, 2026
 4860 views

Quantitative User Testing vs Qualitative

Your funnel report has flagged the same drop for three sprints in a row: shoppers reach the shipping step of checkout and leave. Then your team watches a handful of recorded sessions, and participant after participant scrolls past the delivery estimate looking for a total cost that never appears. One source told you where the problem sits, and the other showed you why it happens, which is the practical difference between qualitative vs. quantitative UX research.

Most product decisions need both kinds of evidence, at different moments, and in different amounts. Below, we compare what each approach measures, how many participants each one needs, and which methods belong where. We finish with a study plan that runs them in sequence and a way to report the results so stakeholders can act on them.

TL;DR

  • Qualitative UX research shows why users struggle by observing a few sessions directly; quantitative UX research measures how many and how much from a larger sample.
  • Nielsen Norman Group (NN/g) suggests about 5 participants per round of qualitative usability testing and about 40 for most quantitative studies.
  • Report small studies as counts with context, since a percentage from 5 or 10 participants carries a wide margin of error.
  • The strongest plans run both in sequence: numbers to locate the problem, sessions to explain it, then numbers again to confirm the fix.

Compare Qualitative vs Quantitative UX Research at a Glance

The clearest way to tell the two apart is how the evidence reaches you. In NN/g’s guide to choosing UX research methods, qualitative studies collect behavior and attitudes firsthand, by watching or listening to participants, while quantitative studies collect them through an instrument such as a survey, a task metric, or an analytics tool.

Qualitative UX researchQuantitative UX research
Question it answersWhy are users struggling, and what would help?How many succeed, how long it takes, how much better the new version performs
DataObserved behavior, think-aloud comments, quotes, video clipsTask success, time on task, error counts, ratings, clicks
Typical sampleAbout 5 participants per roundAbout 40 participants for most studies
Study conditionsFlexible; the researcher can probe and adjustFixed tasks and conditions so results can be compared
AnalysisThemes and findings ranked by severityDescriptive statistics and confidence intervals
Best stageAny stage, especially while the design is still changingA working product or prototype you want to benchmark
Main riskGeneralizing from a few usersKnowing the size of a problem without knowing its cause

The same study script can feed either approach. What changes is the claim you can make at the end, so the choice starts with the decision you need to support.

What Qualitative UX Research Shows

Qualitative UX research explores users’ needs, experiences, and behavior through methods such as interviews, observation, and usability testing. The protocol stays flexible, so a researcher can ask a follow-up question the moment someone pauses, or reword a task that stops producing useful evidence. The output is a set of findings, each tied to the moments that produced it and ranked by how badly the problem blocks the task.

Qualitative testing is also where the quotes and clips come from that make a readout memorable. A short clip of one participant explaining that they expected free shipping until the final screen can account for a drop in conversion faster than a chart can.

Both moderated and unmoderated sessions produce qualitative data. In a moderated session, the researcher guides the participant through the tasks, observes, and asks follow-up questions about what they did and what they expected. In unmoderated usability testing, participants complete the study on their own devices while the session and their think-aloud commentary are recorded, and both formats run well as remote usability testing.

What Quantitative UX Research Measures

A quantitative study runs fixed tasks with enough participants that the numbers describe the wider audience. Behavioral metrics such as task success rate, time on task, and error rate record what participants did. Questionnaires such as the System Usability Scale (SUS) and the Single Ease Question (SEQ) record how easy the experience felt.

A SUS score runs from 0 to 100 but does not represent a percentage, and a score of about 68 is treated as average. Customer Satisfaction (CSAT) and Net Promoter Score (NPS) belong to the same attitudinal family, measuring satisfaction and loyalty, so they sit beside usability metrics rather than standing in for them.

Numbers also need a reference point. NN/g’s comparison of quant and qual usability testing uses a 60% completion rate to make the point: on its own, the figure cannot tell you whether the design is doing well or badly. Quantitative work is therefore usually comparative, setting this release against the last one or your flow against an agreed target.

On the Userlytics platform, quantitative testing covers tree tests, card sorts and surveys along with time-on-task and success or failure measures, with SUS, SEQ and NPS available as templated questions. For a closer look at survey design, analytics and benchmarking, see our guide to quantitative UX research methods.

Separate What Participants Say From What They Do

Qualitative vs. quantitative is one axis. NN/g’s research-methods landscape adds a second one that settles most debates about where a method belongs: attitudinal research captures what users say about their beliefs and preferences, and behavioral research captures what they actually do. The two often disagree, which is exactly why the distinction matters when you choose a method.

Qualitative (observed or heard directly)Quantitative (measured through an instrument)
Behavioral: what participants doModerated usability testing; unmoderated think-aloud sessions; field observationTask success and time on task at scale; tree testing; first-click testing; A/B testing on live traffic; product analytics
Attitudinal: what participants sayUser interviews; open-ended survey questions; diary studies; focus groupsClosed survey questions; SUS and SEQ; NPS and CSAT

Surveys show why the second axis helps. A survey is attitudinal by design, and it becomes qualitative or quantitative depending on its questions: rating scales produce numbers you can compare, while open text produces reasons you have to read.

Two methods change sides depending on how you run them. A card sort with a handful of participants explaining their groupings out loud is qualitative, and the same sort with a large sample analyzed for how often cards were grouped together is quantitative. Unmoderated usability testing works the same way, producing findings from a few think-aloud recordings and metrics from a large sample with fixed tasks.

A tree test is a useful contrast. Participants search a stripped-down, text-only hierarchy of your navigation, so the method is behavioral, and with a large enough sample it reports how many found each item and where the rest went instead.

Place your planned method on both axes before you write the screener. If the cell does not match the claim you need, change the method before you recruit, and browse our overview of UX research methods for alternatives.

Size the Sample to the Claim You Want to Make

Sample size follows from what you plan to say when the study closes. “The delivery cost is hard to find” needs a different base than “the new checkout raised completion.”

Study typeNN/g guidanceWhat it can supportWhat it cannot support
Qualitative usability testingAbout 5 participants per round; 3 to 4 per group when testing two groupsThe main problems in a flow and why they happenHow common each problem is across your audience
Several small qualitative roundsThree rounds of 5 instead of one round of 15Evidence that each fix worked before the next roundA benchmark score
Quantitative usability testingAbout 40 participants for most studiesA success rate within about 15 percentage points of the true rate, at 95% confidenceWhy participants failed
Lower-precision quantitative study15 participants, accepting a 20% margin of error at 90% confidenceA directional read when budget is tightA verdict between two designs that perform closely
Card sortingAt least 15 for a qualitative sort; 30 to 50 for a quantitative oneGroupings participants expect, and with the larger sample, how consistent they areWhether participants can find items in the finished menu

The 5-participant guidance comes from a mathematical model by Jakob Nielsen and Thomas Landauer, which estimated that five participants uncover about 85% of a design’s usability problems, provided the design still has plenty of problems left to find. The same article suggests splitting a budget for 15 participants into three studies of 5, so each round can check the fixes from the round before.

The 40-participant figure comes from a calculation in NN/g’s summary of quantitative sample sizes. To estimate a success or conversion rate within 15 percentage points at 95% confidence, you need 39 participants, which NN/g rounds up to 40.

Larger samples take more recruiting, so many teams reserve quantitative rounds for decisions that genuinely need a number. On the Userlytics platform, those studies can recruit from a panel of 2M+ participants in 150+ countries, or from your own customers through BYOU (Bring Your Own users).

Report Small Studies as Counts With Their Limits

A percentage from a small study looks far more precise than it is. In NN/g’s article on why numbers from qualitative studies mislead, 7 of 10 participants complete a task, yet the success rate for the whole audience could fall anywhere between 39% and 90% at 95% confidence. Flexible protocols add more variation, since the researcher probes and adjusts differently from one session to the next.

Write the count and the base instead. For instance, “4 of 5 participants missed the delivery estimate” tells a stakeholder exactly what the team saw, and one more clause can name the limit: the round documents the problem observed in the sessions and leaves its frequency to a larger study.

Choose the UX Research Method That Fits the Decision

Start with the question the team has to answer, then pick the approach and the method. The table maps common product questions to the evidence that answers them.

The question in front of the teamLeanA UX research method that answers it
Why do users abandon onboarding at the same screen?QualitativeModerated or unmoderated think-aloud sessions on that screen
How many participants can finish checkout without help?QuantitativeUnmoderated usability testing with task success and time on task, about 40 participants
Did the redesign perform better than the version it replaced?QuantitativeThe same tasks and metrics run on both versions
Can users find “Returns” in the new menu structure?Quantitative, then qualitativeA tree test at scale, then sessions on the paths that failed
What do buyers expect a pricing page to answer?QualitativeUser interviews plus think-aloud reviews of the current page
How common is a motivation that came up in interviews?QuantitativeA survey with closed questions built from the interview themes
Which of two concepts should move into design?BothPreference questions at scale plus sessions that capture the reasons
Where should the next research round focus?QuantitativeAnalytics or a benchmark study that shows the weakest tasks

Match the Approach to Your Design Stage

The design stage narrows the choice further. NN/g separates formative research, which shapes a design while it is still changing, from summative research, which assesses a design once it is built.

  • Discovery: interviews and field observation show what users are trying to get done before anything is drawn.
  • Design iteration: short qualitative rounds on clickable prototypes catch the biggest problems while they are still cheap to fix.
  • Pre-launch validation: a quantitative round on the near-final build sets the baseline you will compare against later.
  • Post-launch tracking: repeated benchmarks and analytics show whether the experience holds up from release to release, with qualitative sessions whenever a metric moves.

Qualitative work stays useful at every stage. Its job shifts from shaping the design early on to explaining why a number moved later.

Not sure which study design fits your research question? Userlytics’ UX consultants can plan the study, moderate the sessions, analyze the data, and present recommendations to your stakeholders. Explore UX Consulting

Combine Qualitative and Quantitative UX Research in One Study Plan

Combining the two works best as a sequence, where each round answers the question the previous one raised. Three orders cover most situations, and a single study can also carry both kinds of data.

Start With the Numbers When You Know Where Users Struggle

Analytics or a benchmark study points to the step where completion falls. A qualitative round of about 5 participants on that step shows what they misunderstand or cannot find. Once the fix ships, the same metric tells you whether the change worked.

Start With Sessions When You Do Not Yet Know What to Measure

Interviews or exploratory sessions surface a reason or a workaround you had not expected. A survey with closed questions built from those themes then shows how widespread each one is. The roadmap can weigh them by frequency instead of by how vivid the quote was.

Collect Both Kinds of Data in the Same Study

One study can carry both, with one caution. NN/g’s comparison lists think-aloud under qualitative methodology and notes that quantitative studies usually run without it. Narrating each step slows participants down, so keep timed tasks silent and ask open questions once each task ends.

Moderated sessions can mix the two as well. A researcher can run a card sort inside a live session and then ask why the participant placed certain cards together, which adds the reasons behind each grouping.

For unmoderated studies, Userlytics’ AI Moderation feature adds follow-up questions to the activities you select. You write the objective in plain language; the AI evaluates each response against it and asks a follow-up in the study’s language, and each follow-up response is kept with the activity it came from. The feature brings probing to unmoderated research, while moderated sessions remain the place for open conversation.

See Both Approaches in a Three-Round Model Study

Here is how a team might plan a checkout redesign with both kinds of evidence. The plan is an illustrative model study and reports no results.

RoundApproachSetupWhat it answers
1. BaselineQuantitativeAbout 40 participants, unmoderated, three checkout tasks on a staging site with test payment details; task success, time on task, and SEQ after each taskWhich task performs worst, and by how much
2. DiagnosisQualitativeTwo rounds of about 5 participants with think-aloud, focused on the weakest task, with a fix tested between roundsWhy participants stall, and whether the first fix removes the problem
3. Re-measureQuantitativeAbout 40 new participants, identical tasks, wording, metrics, and staging setupWhether the redesign moved the metric by more than the margin of error

Keep rounds 1 and 3 identical in tasks, wording, and audience, since a change in any of them weakens the comparison. Round 2 is free to adapt from session to session, because its job is understanding.

See How a Real Study Counted Qualitative Evidence

A Userlytics x Sage study on AI visibility and B2B software buyers shows both kinds of evidence in one report.

  • Study design: 60 unmoderated sessions with prospective finance-software buyers, across three markets.
  • Counts reported with their base: Sage showed up in the AI assistant’s first answer in 41 of the 60 sessions and in 47 sessions overall. Its website confirmed or strengthened the AI-formed impression in 45 of 60 visits, or 75%.
  • Observed reasons: The sessions showed where trust broke down, for example, when missing pricing led AI assistants to fill the gap with outside price estimates the buyers had no way to check.
  • A measure the team can repeat: Userlytics assessed each journey with its six-dimension AI Readiness framework and gave Sage a scoring instrument so the research can be rerun over time.

The whitepaper presents its findings as directional, without claiming they represent all B2B software buyers. The counts say how often something happened in this sample, and the observed sessions explain why.

Report Both Kinds of Evidence So Stakeholders Act on Them

A study changes a decision only when the users making it can read the evidence quickly. These steps keep both kinds of findings readable in one report.

  1. Lead with the decision. Open with the question the study answered and what the team should do next.
  2. Include the sample size. Put the number of participants next to each figure, and keep percentages for samples large enough to carry them.
  3. Pair each number with a clip. A short video of one participant hitting the problem gives the metric a face, and the metric shows the clip reflects more than one session.
  4. Rank findings by severity. Look for recurring patterns, and also investigate severe issues encountered by individual participants before deciding what to change.
  5. State the limit. One sentence on what the study cannot show answers the objection before anyone raises it.

On the Userlytics platform, the Qual and Quant AI Reports follow the same split. Qual Reporting turns a completed study’s sessions into themes, key findings, strategic areas, and recommendations with supporting evidence and links to sources across sessions. Quant Reporting compiles the charts from a study’s Metrics tab into a shareable report, pairing each chart with an AI-generated summary.

The researcher reviews what the AI organized, checks the evidence behind it, and decides what to present to stakeholders. During analysis, AI Insights answers plain-language questions about a study, with every answer citing the sessions, timestamps, and annotations it is based on.

Avoid These Mistakes in Qualitative vs Quantitative UX Research

Most problems with mixed evidence come from asking a study for a claim it was never designed to support. Watch for these seven.

  • Generalizing from a small sample. A five-person study can collect quantitative data, but estimates will carry considerable uncertainty. Report the counts and sample size, and avoid presenting them as reliable estimates for your wider audience.
  • Reporting percentages from a handful of sessions. “60% of participants” from a sample of 5 implies a precision it cannot carry; write “3 of 5” and state the limit.
  • Treating every survey as quantitative. Open-ended answers are qualitative data that have to be read and coded, and closed questions only measure the options you thought to offer.
  • Swapping attitudinal metrics for behavioral ones. A rising NPS indicates a more favorable balance of promoters to detractors; however, it does not show whether users can complete a task.
  • Reading SUS as a percentage. A SUS of 80 sits above the average of about 68, and it does not mean the product is 80% usable.
  • Timing tasks while participants think aloud. Narration can slow users down, so collect time on task in silent tasks and gather the reasons afterward.
  • Exclude unusual sessions without watching them. A participant who finishes very fast or fails every task may reveal a real problem, so review the recording first and exclude a session only when the evidence clearly shows the participant ignored the instructions or never engaged, noting why.

Your Next Step in Qualitative vs Quantitative UX Research

Choosing between qualitative and quantitative UX research gets easier once the claim is written down. Before your next planning meeting, pick one decision on the roadmap and write the sentence you would need to say to settle it.

If that sentence contains “how many,” “how much,” or “better than,” plan a quantitative round of about 40 participants. If it contains “why” or “what gets in the way,” start with a qualitative round of about 5, and book the follow-up round now so the first study’s open question already has a slot.

See how one study plan can combine both the participant feedback and usability metrics. Book a free demo.

FAQs

Quick answers to the questions teams ask most about qualitative vs quantitative UX research.

Usability testing can be either, depending on how you run it. A test with a few participants, think-aloud, and a flexible protocol is qualitative. The same tasks run with about 40 participants under fixed conditions, with success rates and task times recorded, make it quantitative usability testing.
Surveys are attitudinal, and they can produce both kinds of data. Closed questions such as rating scales give you numbers to count and compare, and open-ended questions give you answers to read and code, which is qualitative analysis.
Card sorting has both versions. A small sort where participants explain their groupings is qualitative, and NN/g suggests at least 15 participants for it. A quantitative sort looks at how often participants grouped cards together and needs at least 30 to 50 participants.
The two follow different kinds of rigor. Qualitative testing depends on careful task design, consistent probing, and findings traced back to recorded evidence, while quantitative testing adds statistical protection against chance through confidence intervals and significance testing. Each is rigorous for the claim it is built to support.
For many product questions, yes. Descriptive statistics and a confidence interval for each metric cover a typical benchmark study. Bring in a specialist when you compare several designs, segment the sample or need significance tests across groups
Both have a place. Your own customers know the product and its vocabulary, which suits studies of existing features, while panel participants bring fresh eyes and can be screened for the audience you want to reach. On Userlytics, BYOU lets you invite your own participants alongside or instead of the panel.
AI can take on much of the mechanical work, such as transcription, tagging, and a first pass at themes, while the researcher checks the evidence and makes the call. On Userlytics, the AI features run on OpenAI under a Data Protection Agreement and are instructed on each study rather than trained on client data. Treat AI output as a draft to verify against the sessions it cites.

Userlytics

Userlytics
Userlytics Corporation

Since 2009, Userlytics has helped enterprises, government organizations, nonprofits, agencies, and startups understand their users and optimize their user experiences. Through our UX research platform, global participant panel, and educational resources, we make user research more accessible while sharing insights, expert perspectives, and conversations that help shape the UX community.

Schedule a Free Demo

Start improving your UX!

Book a Free Demo:

Related Resources

Blog
April 27, 2026

UX Research Methods: The Complete 2026 Guide

Discover the 12 best UX research methods for 2026. From moderated usability testing to card sorting and tree testing — know which method to use and when.
Blog
January 30, 2025

UI UX Quotes | Einstein’s Strategy to UX: Mastering Qualitative Usability Testing

Explore the benefits of qualitative usability testing in UX. How Einstein's strategy can revolutionize user experience and user satisfaction.
Blog
January 15, 2025

UI UX Quotes | Expect the Unexpected in UX: Haruki Murakami’s Usability Journey

Explore the "unexpected in UX" with us on a usability testing journey, inspired by Haruki Murakami's wisdom.
Blog
September 4, 2025

Why 10+ Participants Give You the Complete UX Picture (Not Just 5)

The 5-user rule is outdated. Learn why 10+ participants give you complete UX insights you can trust and act on.

Didn’t find what you were searching for?

Ready to Elevate Your UX Game?

×Returning client?
Contact us