User Testing Questions

Userlytics enables you to usability test the user experience of your websites and competitor websites, unreleased Mobile Apps, advertisements, and social media content, at all stages of the design and development process.

We capture the interaction, the verbal commentary of the participants, AND the participants themselves and their context through their webcam, so you can be sure they are your target Persona.

We have native applications for Android & iOS for both offline and online/offline mobile testing of apps, prototypes and mobile web browsing. We also offer an alternative methodology that allows the user to natively test the UI on their mobile device while viewing instructions and answering questions on their desktop/laptop computer.

Brand managers, Product Managers, Advertising and Marketing Agencies, User Experience professionals, Marketers, Mobile App and Website Designers and Developers all benefit from the capability to conduct remote usability testing on their prototype websites, mobile apps and digital assets within the context of participant homes and offices.

Yes, one of our senior UX Consultants did an in-depth webinar about this topic. You can see it here

None. You do NOT need to add snippets of code anywhere on your website. You do NOT need to insert an SDK into your Mobile App. You can conduct remote usability testing at any stage of the design and development and launch of your website, mobile app or other digital asset, with no type of intrusive modifications necessary to your proprietary code. Full compatibility with all user experience design, prototyping and prototype delivery platforms

There are a number of different ways you can leverage Userlytics for remote usability testing:

• You can use your own participants, or have them provided by a 3rd party user experience respondent panel, or ask Userlytics to recruit and incentivize the participants, or leverage its over 200,000 now member panel

  You can configure participant redirects to specific landing pages for competes, abandons, etc.
  
  Respondents download a small app that checks if they have the necessary equipment (e.g.: webcams, when applicable) and that takes them through the tasks and questions you have defined

• You can test a website, a wireframe, a set of static images, a video, a powerpoint or Google Doc presentation, or any other type of digital asset accessible from a computer, including desktop applications.
• You can test a mobile App that is available on the Apple Store or Google Play, or an unreleased App using Testflight or similar, on both smartphones and tablets.
• You can simultaneously view the screen interactions, listen to what the participant hears, and see the participants' face and surroundings in a "Picture-in-Picture" (PiP) format video.
• You can use our customizable templates for all types of use cases, work flows and scenarios for easy task and exit survey question formulation.
• You can create an invitation link for participants that forces them to register their name and email, or one that incorporates a unique identifier in the link (e.g.: for use with 3rd party panel providers) which you can send or post anywhere.
• You can use a customizable respondent feedback invitation widget that you can add on any website page.
• You can configure participant redirects to specific landing pages for competes, abandons, etc.

Respondents download a small app that checks if they have the necessary equipment (e.g.: webcams, when applicable) and that takes them through the tasks and questions you have defined.

• You can access the full video and survey question results, share project by project access with clients and colleagues, and download videos, quantitative metrics and survey results for your own reporting format.
• You can access a hyperlinked "Video Highlights" section with key comments and events linked to specific videos and timeline moments.
• You can download specific video clips with or without annotation subtitles.
• You can request an audio transcript of the videos with time stamped marked text for easy searching of videos and links to specific timeline moments.
• You can request an analysis and executive summary of results with detailed recommendations for each issue uncovered."

By projecting the interacting screen of the smartphone or tablet (Android or iOS) onto the participant computer screen and recording the screen and the participant through their webcam we enable a "software only" approach that avoids the need for camera hardware, complex logistics and thus professional usability testers when conducting remote user testing.

This enables conducting unmoderated remote usability tests on Mobile Apps with respondents that match your exact target Persona. Since you can see the participants' face and context and surroundings, you can be sure of the demographic profile of the participant.

See the following blog post on methodologies for remote usability testing on mobile devices.

You can direct participants to perform any goal or task or question, in any order, including with branching logic, that may help you optimize the user experience of your websites, mobile Apps, online advertising and content, and discover the reasons behind your analytics and funnel conversion results. You can also leverage pre-formatted system usability scale questions, and net promoter score questions, and have results automatically tabulated (as well as time on task, success/failure etc)

We provide a set of customizable templates.

You might ask about:

• Site navigation
• Mobile App downloads and interactions
• The understanding of an ad message
• How do customers find your website or App or those of your competition
• The process for satisfying a need or objective
• Ease of use and intuitiveness
• Design, video, colors, layout
• Specific processes such as: registration, log in, check out, utilities, etc.
• If they would use your site or recommend it to a friend
• Other user experience & usability questions specific to your user testing objectives
• Comparisons with competitors, production and prototype assets, offline and online media

Prototype user testing allows you to easily and quickly identify issues with the user interface and user experience of your prototype, mobile App, ad concept, storyboard, or wireframe during the early stages of design or development, and prior to spending significant sums of money on multivariate testing. Early detection of issues provides the opportunity to make necessary course corrections early in the design and development process, when change is less costly.

Early user experience testing of websites and mobile apps gives a better understanding of what features your users will appreciate, how and why they react to your message, and limits problems identified after your next product launch, campaign or upgrade. They also answer the question of "Why" participants engage in the behaviour you detect through analytics.

We can work with any platform you use for regular QA testing of Mobile App prototypes, such as Testflight or similar. We project manage the whole process of setting up the targeted participant profiles and Personas with the software required for conducting the mobile app usability test.

You can user test any type of file (JPG, .GIF, .PNG, PDF, HTML, JS, CSS, etc), or Mobile App (Android, iOS, Free, Paid, released, unreleased...)

If you have a proprietary file format that you are unsure of, please contact us and we will be happy to review your needs and work with you to make your remote usability test project a success.

We have implemented commercially reasonable technical and organizational measures designed to secure uploaded files from accidental loss and from unauthorized access, use, alteration or disclosure. Our participants have signed up to non disclosure clauses, and we can project manage a customized NDA from usability test respondents as part of the test workflow. Another alternative or complement is to set up a test on a password protected client site, with a temporary password provided to user test participants.

For privacy reasons, user experience test panel participants are normally prohibited from completing purchase processes with their own personal credit card information. As an alternative, we can facilitate the use of dummy credit card information for such purposes, including the use of an application that delivers unique codes to each participant.

However, if a real purchase is necessary, we can project manage participant reimbursement if the process of purchasing through the Apple Store or Google Play is part of the remote usability test of a mobile App.

You can run a "preview" of the test flow/journey/script, and then edit your directions, goals, tasks and questions or demographic selections until you are satisfied. You even change the test script or maximum session time or number of respondents in real time after your test has been launched and is live.

If you are using your own usability test participants, it depends on how quickly they respond to your invitation. Our platform processing time takes between 1 to 5 minutes, so you can receive results extremely quickly.

When using the Userlytics Panel, with broad based demographics, after a through QA review of every user test video, results are available in as little as 24 hours, sometimes within hours.

For narrowly based demographic targets, it depends on the filters ands behavioural screener used to set up the project, and thus how narrowly defined the target Persona is."

You may view access your remote user testing results at any time within 90 days after they have been delivered. You can also download the user experience test video results anytime within the 90 day period and keep and edit and share as needed.

Please contact us to learn more.

Pricing for remote user testing depends on the following drivers:

• Usability test participant demographic profile - Target Persona
• Maximum user experience test session time
• Number of user experience test participant completes desired
• Whether usability test design - scripting is required
• Whether user test review, analysis and recommendations are required

Pricing can be annual-licensing based or Ad-Hoc, on a per usability test project basis.

Contact us for information on how we can help you achieve your unmoderated user experience testing goals.

If you have a DIY account, then you would pay for the credits when you go to launch the test, after you have fully setup the test.

If you have a Custom Project or Enterprise Plan, then you would purchase the credits up front and they would be available in your account for when you look to run your upcoming test(s).

You need to pay in US dollars using a Credit Card or PayPal. For custom projects wire transfers or checks may also be used.

We can conduct user testing with the maximum session time set to 5, 10, 15, 20, 30, 60 minutes or any maximum duration you choose.

However, if possible we recommend keeping usability test sessions designed for 20 min session times on average, so as to capture the outliers with sessions set to a maximum of 30 minutes.

We recommend using Microsoft Internet Explorer 8 and higher, Mozilla Firefox 4.x and higher, Apple Safari 4.x and higher, Google Chrome 4.x and higher.

Yes, you may invite participants from our Global Participant Panel, or use a 3rd party panel provider, or invite participants yourself by sending them invitation links or posting them on your website or social media sites. Contact us for a customized recruitment quotation from anywhere in the world, at any incidence level.

Experts generally agree that using 5-10 participants per user test iteration will uncover 80% of usability issues, and is the most efficient manner to conduct agile user experience testing. We have found that it is more efficient and cost effective to conduct numerous small sample studies iteratively, and then use multivariate testing at the end of the process with larger samples to conduct final optimization. Please refer to the following reference materials:

• March 2000: Why You Only Need to Test with 5 Users
• March 2003 : What's in a Number?
• September 2006: How Many Usability Test Participants do you Need?

You can filter our remote user testing panel for education, age, gender, employment status, household income, education, civil status and similar. In addition, you may define a behavioural screener to select usability test participants based on highly specific demographic profile criteria, both B2B and B2C, from around the world to target your specific target Person for your remote usability test.

The Userlytics user experience test panel comprises crowd-sourced respondents from all backgrounds. However, most of our clients ask us to custom recruit highly specific demographic participants from around the world.

Our remote user testing participants are rated by our clients and we use those ratings as a selection basis for remote user test projects. In some cases our clients ask us to recruit first time usability test participants.

Yes, you can! If you rate a result as a "1 Star", within 7 days of receiving the result (or simply let operations@userlytics.com know), we will conduct a second in-depth review. We will replace the result if:

• Participant was not following a "Think Aloud" protocol
• Participant was not making a good father effort to follow instructions
• There were technical issues with the recording

If however, the instructions were confusing (and you had not taken advantage of our optional Professional Services and have us design them), or, there was a bug in the test asset you directed participants to, then participants will be paid their incentives, and the credit(s) used will not be replaced (We can however still delete the result from your account if you wish)

Yes. Participants can reuse the invitation link multiple times. It will generate a different code each time, which can be used each time with the Userlytics Recorder

• Capture of computer or mobile device screen interactions;
• Capture of participant audio commentary;
• Capture of participant and context;
• Integrated "Picture-in-Picture" (PiP) video of participant, verbal "think aloud" commentary, and screen device interactions;
• Branching logic;
• Configurable testing methodology (webcam, screen, audio recording + metrics, or screen, audio recording + metrics or audio recording + metrics)
• Personal Identifiable Information (PII) protection;
• User testing of production sites, mobile apps and desktop applications;
• UX testing of prototypes, wireframes, PNG sketches, unreleased mobile Apps;
• Usability testing of Android or iOS, free or paid, production or prototype mobile apps;
• In-linetime stamped hyperlinked annotations;
• Downloadable videos (MP4), excel data (time on task, success/failure rates, System Usability Scale, Net promoter Score and other quantitative data);
• Downloadable video clips with annotation subtitling;
• Highlight reels;
• Qualitative usability testing session audio transcription;
• Usability Test Design, Scenario - Script preparation;
• User Experience Test review and analysis and executive Summary report with recommendations;
• User test respondent recruitment and incentives with a global panel of almost 200,000;
• 3rd party user testing respondent panel integration;
• White labelling - private labelling of dashboard results;
• Project management.

Mac, Windows, iOS, Android....We cover all the main operating systems. And you can use any prototyping platform such as Proto, Axure, Invision, Justinmind, Adobe XD...

You can annotate each user experience test video with key events and comments, share an aggregate list of annotations with each one hyperlinked to the relevant video and timeline moment.

It refers to a recording of the webcam view of the usability test participant, together with the recording of the screen the participant is interacting with, as well as the audio "think aloud" commentary of the user testing participant and the UX moderator, if any.

This unique feature enables the blocking of screen recording during sensitive moments in the test journey, to protect personal identifiable information, while continuing to record the webcam view of the participant and their "think aloud" commentary.

Yes. You can create and download individual video clips, insert annotations as subtitles, and automatically create and download a highlight reel.

Yes. You can have them transcribed, search for key terms, and click on the results to go directly to the relevant video and timeline moment

Yes it is possible, however, a test with children requires certain protocols to be met in order to ensure all guidelines and laws are being followed. We have created a guide for this purpose here.