A shopper picks a rain jacket, chooses her size, and reaches the last step of checkout. The delivery fee appears, she pauses for a few seconds, and the tab closes. Your analytics records the drop on that step, but it cannot say whether the fee, the delivery date, or a form error sent her away.
Ecommerce usability testing closes that gap. You get to watch shoppers attempt realistic shopping tasks and hear what they think at the moment they hesitate. Here is how to pick the method, write the tasks, recruit the right people, and turn sessions into fixes.
TL;DR
- Ecommerce usability testing explains why shoppers stall on a step your analytics already flags.
- Match the method to the journey stage, from tree tests for navigation to moderated sessions for checkout friction.
- Give shoppers a need and a budget instead of a named product, so they browse the way they would at home.
- Five comparable shoppers per round surface most qualitative problems; plan for about 40 when you need metrics to benchmark.
- Fix what you find and test again, because three small rounds teach you more than one large study.
Use Ecommerce Usability Testing to See What Analytics Misses
Ecommerce usability testing puts representative shoppers in front of your store, gives them realistic shopping tasks, and records what they do and say. Your analytics show what happened on a page. A usability test shows why it happened, which is the part your team needs before it decides what to change.
The scale of the problem is well documented. Baymard Institute calculates an average cart abandonment rate of 70.22% across 50 studies. Some of that is window shopping: in the same survey, 42% of US online shoppers said they had left a cart because they were just browsing.
The remaining reasons are where testing earns its place. Set the browsers aside, and Baymard’s survey of US shoppers breaks down like this:
Several of these are due to design and content problems that a test can catch before launch: an account wall, a total that only appears on the final screen, a returns policy nobody can find. Baymard estimates that the average large ecommerce site could lift its conversion rate by 35.26% by fixing checkout design issues alone.
Analytics points you to the step. A usability test tells you which of these reasons is at work on your own store.
Map Each Shopping Stage to the Right Ecommerce Usability Test
Most teams limit ecommerce usability testing to the checkout and stop there. Plenty of problems start earlier, since a shopper who never finds the right category never reaches the cart.
Start with the stage your analytics flags first, then work back one step. A drop at checkout sometimes begins with a product page that left a question open.
Choose a UX Research Method That Fits the Question
Each UX research method answers a different kind of question, so choose based on what you need to learn next.
Moderated Sessions for Checkout and High-Stakes Flows
In a moderated session, a researcher guides the session, observes the shopper completing tasks, and asks follow-up questions to understand their actions and expectations.. Use moderated UX research sessions when you need to understand hesitation, for example, why someone stops at the payment step or reads the returns policy twice.
Moderated sessions require a facilitator and coordination with participants, making them more time-intensive to organize and run than unmoderated studies, but they add a layer of context that is unmatched.
Unmoderated Studies for Product Pages and Larger Samples
In unmoderated usability testing, participants complete tasks on their own devices while the session records their screen and voice. You get more sessions, faster and at lower cost, which makes this the default for product listing and product page questions. Our unmoderated ecommerce template gives you a starting script for a first study.
The limit is depth, since nobody is there to ask why a shopper hesitated. Luckily, Userlytics’ AI Moderation feature narrows that gap: you write the objective for an activity in plain language, and the AI generates relevant follow-up questions on the spot, in the study’s language. You choose which activities need the extra probing, and each follow-up answer stays linked to the activity that prompted it.
Tree Testing and Card Sorting for Navigation
Tree testing strips away the visual design and asks shoppers to find items in a text-only version of your category structure. If shoppers struggle to find rain jackets in the tree test, revisit the category labels and structure before testing the full navigation design.
Card sorting works earlier in the process. Open card sorting shows how shoppers would group your products themselves, while closed card sorting checks a structure you already have. For the design side of the same problem, our guide to improving ecommerce findability covers site search, related products, and cross-device behavior.
First-Click and X-Second Tests for First Impressions
A first-click test records where shoppers click first when given a task, which shows whether their instinct lands in the right place. An X-second test shows a page for a few seconds, then asks what people remember.
In other words, first-click testing evaluates whether shoppers recognize where to start, while X-second testing assesses initial impressions and recall.
Comparative Tests for Design Decisions
Comparative testing settles arguments that opinion cannot. A filter study for Duluth Trading Company shows how.
The Duluth case study explains how the team scaled research without adding headcount.
Keep preference and performance apart in your report. A layout people say they like is a different finding from a layout they use faster.
Remote Usability Testing Versus an In-Person Lab
Remote usability testing lets shoppers use their own phone or laptop at home, with their own saved addresses, browser habits, and distractions. For most ecommerce questions, that is closer to real shopping than a lab, and it lets you recruit from more regions at once.
An in-person ‘lab’ still has a place when you test something physical, such as a self-checkout kiosk or an in-store app used next to the shelf. For a website or shopping app, remote sessions are usually the faster and more representative option.
Accessibility Testing for Shoppers With Low Vision
A product page with low-contrast prices or tiny buttons can stop shoppers with low vision long before they reach the cart. Give these participants the same purchase tasks as everyone else, and use WCAG 2.2 as the reference for what to fix.
Userlytics supports accessibility studies with participants who have mild to moderate low vision, in moderated or unmoderated sessions, and with severe visual impairment in moderated sessions. The platform is not compatible with screen readers or other assistive devices, so check screen-reader support for your checkout as a separate audit.
Most ecommerce teams need two or three of these methods at a time, matched to the stage they are working on. Let the question choose the method, and let your data choose the question.
Need help scoping your first ecommerce study? Our UX consultants can design the study, recruit the right users, and deliver a findings report. Explore UX Consulting
How to Run an Ecommerce Usability Test
The six steps below take you from a question in your funnel report to a fix you can ship and test again.
1. Start From a Business Question and Your Analytics
Pick one stage and one question your team needs answered, such as why mobile shoppers leave the delivery step more often than desktop shoppers. Pull the funnel data first so the study focuses on the step that matters.
Write down the decision the study will inform. “Should we show delivery costs on the product page?” gives a study a clear job, while “improve checkout” gives it nothing to answer.
2. Write Shopping Tasks That Feel Real
Nielsen Norman Group’s guidance on writing task scenarios recommends tasks that are realistic, ask for an action, and avoid describing the steps. For a store, that means handing shoppers a need and a constraint instead of a product name.
Keep each study to a handful of tasks. A long list tends to produce rushed, careless sessions near the end.
3. Recruit Shoppers Who Match Your Customers
Screen for the behavior that matters: people who have bought in your category online recently, on the device you are testing. Someone who has never bought furniture online will struggle with a sofa configurator for reasons your real customers would not.
For qualitative rounds, Jakob Nielsen’s 2000 analysis at Nielsen Norman Group found that a first round with five comparable participants found about 85% of the usability problems. When a site serves distinct groups, the same analysis recommends three to four people from each of two groups, or three from each of three or more. Mobile and desktop shoppers, or first-time and returning customers, can behave differently enough to treat as separate groups.
Userlytics’ global panel includes 2M+ participants in 150+ countries and 60+ languages, with screeners for recruiting participants who match your buyer profile.
4. Prepare the Store so Shoppers Can Finish the Task
Use a staging environment with test payment details so participants can complete checkout without making a real purchase. If testing on your live store, instruct participants to stop before placing the order. Load realistic products, prices, and delivery options. A half-built product page produces feedback about missing content instead of feedback on the design.
Decide in advance where each task ends. If participants cannot pay, stop the task at the order review screen and say so in the instructions.
Match the devices to your traffic. A desktop-only study will miss mobile keyboards, autofill behavior, and buttons that sit out of thumb reach.
5. Run Sessions and Ask About Moments of Hesitation
Ask participants to think aloud as they shop, and watch for the pauses: rereading a delivery estimate, scrolling back to a size chart, hovering over the pay button. Those moments mark the questions your store left unanswered.
In moderated sessions, probe with neutral questions such as “What were you looking for just then?” instead of “Was the delivery cost confusing?” In unmoderated studies, add a short question after each task, or turn on AI follow-up questions for the activities where you most need the reason behind a response.
6. Analyze Patterns, Prioritize and Retest
Look for recurring patterns, while also investigating severe issues encountered by individual participants. A single session can reveal valuable insights, but teams also need to identify patterns across the study before deciding what to change.
AI Insights speeds up that pattern hunt. You can ask a question about the whole study, and every observation comes with the session and timestamp it came from, so you can watch the clip before you act on it.
Rank each issue by how many participants hit it and how badly it blocked the task, then fix the ones that stopped a purchase first. Nielsen’s same analysis argues for spending a 15-person budget on three rounds of five, since the second round checks the fixes and catches most of what the first round missed.
Turn Findings Into Fixes Your Stakeholders Will Fund
A findings deck that lists 40 issues rarely gets funded. Your merchandising, engineering, and finance colleagues need to see the few problems that cost sales, and they need to see real shoppers hit them.
Group each issue with its evidence and its likely effect on revenue:
The rows show the format with common checkout issues; fill the evidence column with the exact counts from your own sessions. A line such as “four of five mobile shoppers went back to the cart” persuades more than a general claim that people found it confusing.
Lead with a short clip of the worst moment for each top issue. A 30-second video of a shopper giving up does more in a planning meeting than a paragraph of findings, and it keeps the conversation on what shoppers did.
Close the report with the decision you need, the fix you recommend, and how you will test it. The study then becomes the first step of a plan your stakeholders can approve.
Measure What Shoppers Do Alongside What They Say
Behavior tells you whether the store works, and attitude tells you how it felt. Report both, and keep them labeled.
A SUS score is not a percentage; convert it to a percentile ranking against a benchmark before you report it. On the System Usability Scale, a score above 68 is generally considered above average, so a 75 means above-average perceived usability rather than “75% usable.”
Net Promoter Score (NPS) and Customer Satisfaction (CSAT) measure loyalty and satisfaction. They belong in your customer program, and neither can tell you whether a shopper was able to finish checkout.
If you want numbers to benchmark over time or to compare two designs, plan for a larger sample. NN/g’s summary of quantitative sample sizes by Raluca Budiu and Kate Moran recommends 40 participants for most quantitative studies, with fewer acceptable in some cases.
Check That Product Pages Confirm What AI Assistants Tell Shoppers
More shoppers now ask an AI assistant what to buy before they reach your site. Your product page then has to confirm what the assistant told them.
What Userlytics found with Sage
- Who took part: 60 prospective B2B software buyers, 20 in each of the United States, United Kingdom, and France.
- What they did: Researched finance software with their own AI assistants, then visited vendor websites to check the recommendations.
- The named idea: The white paper on the AI-shaped buyer journey calls it the “AI Tax.” When a website leaves a question such as price unanswered, the assistant fills the gap, and the brand absorbs the lost trust if that answer is wrong.
- The limit: The study covered software buyers, so treat it as directional for retail.
How to run the same check on your store
- Recruit shoppers who already use an AI assistant to research purchases.
- Ask them to research a product in your category with that assistant, in their own words.
- Send them to your product page to check what they were told about price, delivery, and returns.
- Note where their confidence drops, and fix that part of the page first.
Avoid These Mistakes in Ecommerce Usability Testing
Most weak studies fail in the setup, long before anyone watches a session.
Frequently Asked Questions
Short answers to the questions teams ask most when they plan their first round of ecommerce usability testing.
Start With the Step Where Shoppers Leave
Open your funnel report this week and find the step with the steepest drop. Write two shopping tasks that walk a shopper through it, recruit five people who bought in your category recently, and watch where they hesitate.
Fix the biggest problem you see, then run the same tasks with five new shoppers. The second round tells you whether the fix worked.
See your checkout through a shopper’s eyes. Plan your first ecommerce usability study with the Userlytics team. Book a free demo
