User Testing FAQ

We have a unique mobile usability testing system that is software only based, so you do not have to worry about being stuck with professional testers and can target your exact demographic profile.

Our system projects the interacting Android or iOS device screen onto the computer of the participant where the screen is recorded together with the webcam view of the participant.

You can usability test for the user experience during mobile based browsing, Mobile Apps that are available in the Apple Store or Google Play, and unreleased Apps.

Userlytics enables you to usability test the user experience of your websites and competitor websites, unreleased Mobile Apps, advertisements, and social media content, at all stages of the design and development process.

We capture the interaction, the verbal commentary of the participants, AND the participants themselves and their context through their webcam, so you can be sure they are your target Persona and not a simply professional tester.

• Capture of computer or mobile device screen interactions;
• Capture of participant audio commentary;
• Capture of participant and context;
• Integrated "Picture-in-Picture" (PiP) video of participant, verbal "think aloud" commentary, and screen device interactions;
• Production sites and mobile apps;
• Prototypes, wireframes, PNG sketches, unreleased mobile Apps;
• Android or iOS, Free or Paid;
• In-line video result annotation;
• Exit survey;
• Usability Test Design, Scenario - Script preparation;
• User Experience Test review and analysis and executive Summary report with recommendations;
• User test respondent recruitment and incentives;
• 3^rd party user testing respondent panel integration;
• White labelling - private labelling of dashboard results;
• Downloadable "Picture-in-Picture" videos;
• Project management";

Brand managers, Product Managers, Advertising and Marketing Agencies, User Experience professionals, Marketers, Mobile App and Website Designers and Developers all benefit from the capability to conduct remote usability testing on their prototype websites, mobile apps and digital assets within the context of participant homes and offices.

None. You do NOT need to add snippets of code anywhere on your website. You do NOT need to insert an SDK into your Mobile App. You can conduct remote usability testing at any stage of the design and development and launch of your website, Mobile app or other digital asset, with no type of intrusive modifications necessary to your proprietary code.

There are a number of different ways you can leverage Userlytics for remote usability testing:

• You can use your own participants, have them provided by a 3rd party user experience respondent panel, or ask Userlytics to recruit and incentivize the participants, or use its over 30,000 member panel.
• You can test a website, a wireframe, a set of static images, a video, a powerpoint or Google Doc presentation, or any other type of digital asset accessible from a computer, including desktop applications.
• You can test a mobile App that is available on the Apple Store or Google Play, or an unreleased App using Testflight or similar, on both smartphones and tablets.
• You can simultaneously view the screen interactions, listen to what the participant hears, and see the participants' face and surroundings in a "Picture-in-Picture" (PiP) format video.
• You can use our customizable templates for all types of use cases, work flows and scenarios for easy task and exit survey question formulation.
• You can create an invitation link for participants that forces them to register their name and email, or one that incorporates a unique identifier in the link (e.g.: for use with 3rd party panel providers) which you can send or post anywhere.
• You can use a customizable respondent feedback invitation widget that you can add on any website page.
• You can configure participant redirects to specific landing pages for competes, abandons, etc.

Respondents download a small app that checks if they have the necessary equipment (e.g.: webcams, when applicable) and that takes them through the tasks and questions you have defined.

• You can access the full video and survey question results, share project by project access with clients and colleagues, and download videos and survey results for your own reporting format.
• You can access a hyperlinked "Video Highlights" section with key comments and events linked to specific videos and timeline moments.
• You can request a full audio transcript of the videos with time marked text for easy searching of videos.
• You can request an analysis and executive summary of results.

By projecting the interacting screen of the smartphone or tablet (Android or iOS) onto the participant computer screen and recording the screen and the participant through their webcam we enable a "software only" approach that avoids the need for camera hardware, complex logistics and thus professional usability testers when conducting remote user testing.

This enables conducting unmoderated remote usability tests on Mobile Apps with respondents that match your exact target Persona. Since you can see the participants' face and context and surroundings, you can be sure of the demographic profile of the participant (and even ask them within the instructions to prove they are who they say they are!)

See the following blog post on methodologies for remote usability testing on mobile devices.

You can direct participants to perform any goal or task that may help you optimize the user experience of your websites, mobile Apps, online advertising and content, and discover the reasons behind your analytics and funnel conversion results. Additionally, you have the opportunity to set up post task exit survey questions for the participant to answer when the goal portion of the test is completed.

We provide a set of customizable templates.

You might ask about:

• Site navigation
• Mobile App downloads and interactions
• The understanding of an ad message
• How do customers find your website or App or those of your competition
• The process for satisfying a need or objective
• Ease of use and intuitiveness
• Design, video, colors, layout
• Specific processes such as: registration, log in, check out, utilities, etc.
• If they would use your site or recommend it to a friend
• Other user experience & usability questions specific to your user testing objectives
• Comparisons with competitors, production and prototype assets, offline and online media

Prototype user testing allows you to easily and quickly identify issues with the user interface and user experience of your prototype, mobile App, ad concept, storyboard, or wireframe during the early stages of design or development, and prior to spending significant sums of money on multivariate testing. Early detection of issues provides the opportunity to make necessary course corrections early in the design and development process, when change is less costly.

Early user experience testing of websites and mobile apps gives a better understanding of what features your users will appreciate, how and why they react to your message, and limits problems identified after your next product launch, campaign or upgrade. They also answer the question of "Why" participants engage in the behaviour you detect through analytics.

We can work with any platform you use for regular QA testing of Mobile App prototypes, such as Testflight or similar. We project manage the whole process of setting up the targeted participant profiles and Personas with the software required for conducting the mobile app usability test.

You can user test any type of file (JPG, .GIF, .PNG, PDF, HTML, JS, CSS, etc), or Mobile App (Android, iOS, Free, Paid, released, unreleased...)

If you have a proprietary file format that you are unsure of, please contact us and we will be happy to review your needs and work with you to make your remote usability test project a success.

We have implemented commercially reasonable technical and organizational measures designed to secure uploaded files from accidental loss and from unauthorized access, use, alteration or disclosure. However, we cannot guarantee that unauthorized third parties will never be able to defeat those measures or use your files for improper purposes. Because respondents are accessing the tests from their homes or workplaces, we cannot guarantee that they will not copy them or use them improperly. We CAN request an NDA from usability test respondents as part of the test workflow. Another alternative is to set up a test on a password protected client site, with a temporary password provided to user test participants.

For privacy reasons, user experience test panel participants are prohibited from completing purchase processes with their own personal credit card information. However, we can facilitate the use of dummy credit card information for such purposes.

We can also manage participant reimbursement if the process of purchasing through the Apple Store or Google PLay is part of the remote usability test of a mobile App.

You can edit your directions, goals, tasks and questions or demographic selections until you are satisfied. You can change the target URL or tasks/goals even after you have received initial results; from the time of the change onwards, new usability test respondents will use the new URLs and tasks/goals.

We can also manage dry-runs to tweak the initial usability test script design.

If you are using your own usability test participants, it depends on how quickly they respond to the invitation. Our platform processing time takes between 10 and 20 minutes, so you can receive results extremely quickly.

When using the Userlytics Panel, after a through QA review of every user test video, results are usually available in as little as 24 to 48 hours.

For custom recruiting usability test projects, it depends upon the demographic criteria and target Persona.

You may view access your remote user testing results at any time within 90 days after they have been delivered. You can also download the user experience test video results anytime within the 90 day period and keep and edit and share as needed.

You can annotate each user experience test video result inline and share with colleagues a hyperlinked lists of annotations.

Yes. You can conduct remote usability testing with participants anywhere in the world, as long as they have a decent broadband connection. Please contact us for an unmoderated usability test proposal in your desired location.

Yes. Please contact us to learn more.

Experts generally agree that using 5-10 participants per user test iteration will uncover 80% of usability issues, and is the most efficient manner to conduct agile user experience testing. We have found that it is more efficient and cost effective to conduct numerous small sample studies iteratively, and then use multivariate testing at the end of the process with larger samples to conduct final optimization. Please refer to the following reference materials:

• March 2000: Why You Only Need to Test with 5 Users
• January 2003: What's in a Number?
• September 2006: How Many Usability Test Participants do you Need?

Pricing for remote user testing depends on the following drivers:

• Usability test participant demographic profile - Target Persona
• Maximum user experience test session time
• Number of user experience test participant completes desired
• Whether usability test design - scripting is required
• Whether user test review, analysis and recommendations are required
• Whether project management of the usability testing process is required
• Whether remote mobile App user testing is part of the process
• Whether participant verbal commentary transcription is required

Pricing can be annual-licensing based or Ad-Hoc, on a per usability test project basis.

Contact us for information on how we can help you achieve your unmoderated user experience testing goals.

You need to pay in US dollars using a Credit Card or PayPal. For custom projects wire transfers or checks may also be used.

The Userlytics user experience test panel comprises crowd-sourced respondents from all backgrounds. However, most of our clients ask us to custom recruit highly specific demographic participants from around the world.

We rate our remote user testing participants and use those ratings as a selection basis for remote user test projects.

However, some of our clients ask us to recruit first time usability test participants. This is why we use a software only solution for mobile app remote usability testing.

You can filter our remote user testing panel for education, age, gender, employment status and civil status. In addition, you may request us to select usability test participants based on highly specific demographic profile criteria, both B2B and B2C, from around the world to target your specific target Person for your remote usability test.

We can conduct user testing with the maximum session time set to 5, 10, 15, 20, 30, 60 minutes or any maximum duration you choose.

However, if possible we recommend keeping usability test sessions in discrete modules of 20 min or less with highly specific use cases and processes to be tested for usability and user experience.

We encourage you to utilize our rating system to help us improve the quality of our service over time. If for some reason a participant has not met your expectations, please contact us and let us know and we will replace it. We value your input and are always looking for ways to improve our service.

Yes, you may invite participants from our UK or Canada-based Userlytics Panel, or use a 3rd party panel provider, or invite participants yourself by sending them invitation links or posting them on your website or social media sites, or you can contact us for a customized recruitment quotation from anywhere in the world, at any incidence level.

We recommend using Microsoft Internet Explorer 8 and higher, Mozilla Firefox 4.x and higher, Apple Safari 4.x and higher, Google Chrome 4.x and higher.

Yes. Participants can reuse the invitation link multiple times. Each time the Userlytics Recorded is downloaded, an image of the Java applet will remain on the participant desktop. This can be eliminated, and will no longer work for that participant testing instance. However, the link the participant was sent/provided for the test can be reused multiple times, and each time they will be asked to download the Userlytics Recorder, an applet that automatically deletes itself from the cache after each test.