Skip to content

Quantitative vs Qualitative User Experience Research

 By Userlytics
 Dec 15, 2017
 1709 views
Home  »  Blog   »   Quantitative vs Qualitative User Experience Research

The conflict between qualitative and quantitative user experience research methodologies has its origin in a time when computers were only a dream, and with the emergence of the social sciences over a century ago. It lies in a struggle that is much older than the field of Usability, User Testing, and Design Research.

quantitative and qualitative testing

And it is still unresolved. The question arises in almost every company, every UX study, every project. Which path to take, which approach is more preferable for the stakeholders, management, and usability researchers? Which one will enable low hanging fruit and which will bring strategic advances?

“I don’t believe in a quantitative approach”, “We need to quantify this”, “Can we put this in numbers?”, “I only do qualitative interviews”, – are the phrases one hears quite often. But, as we know from everyday experiences, the “Divide and conquer” approach is not helpful in solving problems. And it is not a good way to do research, either. What is the best way to avoid going from one extreme to another?

First of all, lets define the terms and avoid semantics. What do we mean when talking about quantitative or qualitative user experience and usability research methods? Is it all about just adding numbers to reports, proper sampling or opportunities to observe user’ needs? When someone announces their love for a quantitative approach, do they mean proper statistics sampling, accurately measuring time on task and mapping click paths? Do countless interviews with users imply a preference for a qualitative user testing methodology?

The difference between the approaches chosen should reside in the answers each type of usability study, quantitative or qualitative, can help you obtain. Quantitative UX research methods can answer the questions of what, how many, & how much. They will help you compare interfaces, features, navigation paths, or study conditions, as well as differences between user testing Personas.

Quantitative research. Quantitative vs Qualitative UX Research

Qualitative methodologies (observations, interviews, contextual inquiries), on the other hand, answer the question of Why? and can drive insights for further user research. They can help UX researchers analyze the reasons for the paths users choose, the shortcuts they use, and the hacks they leverage while listening to their train of thought.

While quantitative methodologies are meant to provide researchers with objective, measurable results, when applied to real contexts, they are not free from subjective interference. Hundreds of books and articles have been written on statistic sampling and formulating questions for a survey (some companies even conduct meta-studies to find out which questions get a better response rate).

But at a certain point of a user experience testing study we are forced to ask only one question and provide only one set of answer choices from this particular sample. And there is no chance to go back and ask the question again, or to change it. Thus it is very important to conduct pilot testing of every part of a UX study, especially when conducting unmoderated online usability studies.

Conversely, it is possible to enrich qualitative UX research methodologies with quantitative usability metrics, using word counts, clicks, task times, errors, SUS scores, etc.

Qualitative research

Quantitative vs Qualitative UX Research

Almost all user researchers are employing both approaches, quantitative and qualitative user research, no matter what their “official” preference or title. Which is a good thing, especially when there are other limitations to the user experience research process.

Online unmoderated usability studies offer a lot of opportunities in terms of saving resources and time: a user experience testing researcher needs to simply define the UX test script, with the instructions, questions, and branching logic, launch the project to as many participants as needed, in as far away locations as necessary, and receive the results within hours.

It is therefore highly scalable, allowing a significant number of tests managed by a single usability researcher. It does not allow for the personalization of a moderated usability study (whether conducted in person or remotely), however, using branching logic it is possible to make an unmoderated UX test study highly personalized to the actions and responses of each individual participant.

Ultimately, the decision on which user testing method to employ (quantitative vs qualitative, moderated vs unmoderated) and how to adjust them for each research project ultimately hinges on the question usability researchers need to always address:

Qualitative research. Quantitative vs Qualitative UX Research

Quantitative vs Qualitative UX Research

The goal of the user experience testing project:

How much it affects our business metrics?

Or

How many users are affected by it?

Or

Why it happens?


Interested in UX Testing?

Data Visualizations


Latest Posts

Blog
September 30, 2024

UX Checklist for Websites: Crucial Steps Before Going Live

Optimize your site before launch with our detailed UX checklist for websites. Learn how UX research can make your website successful.
Read More
Webinar - ULX SWOT Analysis & UX Metrics
Webinar
October 10, 2024

Bridging the Gap: Leveraging Market Research Insights in UX Research

Join our webinar Leveraging Market Research Insights in UX Research and discover how it can drive to better product design.
Read More
Whitepaper
March 5, 2024

The State of UX in 2024

Discover 'The State Of UX In 2024' report: Key insights on UX research evolution, roles of product managers, and future trends.
Read More
Why CEOs Need To Care About UX – Before It’s Too Late
Podcast
September 10, 2024

Why CEOs Need To Care About UX – Before It’s Too Late

Why CEOs Need To Care About UX - Before It's Too Late. Interview with Userlytics' Founder & CEO Podcast
Read More

Didn’t find what you were searching for?

Ready to Elevate Your UX Game?