Mastering the Art of Moderation: Best Practices for UX Researchers

FREE WORKSHOP

Date: January 29th
Time:  12:00 PM EST

This interactive 60-min workshop is designed to level up your live interview skills! You’ll discover proven techniques for building rapport, asking the right questions, and managing interviews like a pro. Through hands-on breakout sessions, you’ll gain practical experience and actionable strategies to make every interview count. Whether you’re a seasoned researcher or just starting out, this workshop will equip you with the confidence and tools to uncover richer insights. 

Don’t miss out—join us to master the art of moderation! As an attendee, you’ll get access to an exclusive 20% discount on any UXC moderation service for a limited time.

What You Will Learn:

• What Makes Moderated Research Effective: Understand what sets moderated research apart from other UX research methods.
• Effective Strategies and Techniques: Learn real-world techniques to properly prepare, execute, and analyze live interviews.
• Live Q&A with an Industry Expert: Get your questions answered in real-time with a senior UX researcher from the Userlytics team.

Who Should Attend?

This webinar is designed for new and experienced professionals involved in both UX and market research, including:

• UX Researchers
• Market Researchers
• Product Managers
• Marketing Professionals
• Business Analysts
• Anyone interested in conducting live participant interviews

Key Discussion Points:

• The Value and Role of Live Interviews in UX Research: Where moderation fits into the broader user research toolkit.
• Best Practices for Effective Moderation: Essential techniques for pre-interview preparation, building rapport, active listening, managing time, and maintaining neutrality during live interviews.
• Hands-On Moderation Practice: Participate in an interactive breakout session to practice live interview techniques, receive peer feedback, and identify areas for improvement in a safe, collaborative environment.

Transcript:

Welcome everyone, Nate from Userytics here. Really excited to kick off this workshop today. See lots of people are starting to join. All right, Carlos, let’s go ahead and get this kicked off. Gonna go ahead and get started here. Again, today, really, really excited. We have a really fun planned workshop for you. that’s going to be quite interactive and actually give you guys a chance to do a little bit of moderation practice yourself.

But before we jump in there, just wanted to quickly introduce myself and my colleague. So I’m your host today, Nate from Userlytics, Head of Channel Development, and just going to be hosting this workshop and kind of guiding everyone through it. But the real expert today is my good friend and colleague, Carlos. Carlos, appreciate you joining.

Thank you, Nate. Thank you for having me today. Well, I’m Carlos. As Nate said, I’m a senior UX researcher at Userlytics. And a bit about my background. So I have a degree in political science and sociology, with special focus in social science. But I also have extensive background with statistical analysis. And for the past three, four years, I’ve been specializing in online research, remote interviews, remote research. So yeah, that’s a bit about myself.

Yeah, and I’ve seen Carlos’s work in action been able to sit in on a few of the sessions he’s done, really an expert in moderation. So I’m glad that we have him here today to kind of give us some good insights to hopefully help all of you who are watching today. Just give you some good tactical strategies that you can use when you’re in your own live interviews.

So before we jump in, we’re going to go ahead and do a poll. So what I’d like to do is get that poll rolling. Let me go ahead and launch it really quick so that everyone in the audience can go ahead and answer that. Yeah, go ahead and put that in. Again, we have a good amount of people, so I’ll wait till they’ll start filling in, but we’re just interested to kind of understand the level of experience for different people in the group today. How often are you doing moderated sessions yourself? Carlos, what about yourself? How often would you say you’re doing these kind of live moderated sessions?

So, me myself, I tend to at least like every couple of weeks have some type of moderated projects. I usually enjoy doing moderated projects rather than on moderator or quantitative surveys. But yeah, yeah, I would say every couple of weeks I tend to other all my interviews and do some moderations.

Very cool. Well, it looks like we got about 80% a few more trickling. I’m gonna go ahead and end the poll so we can see. So if you haven’t yet, go ahead and answer now. I’m gonna go ahead and end it and share the results. Carlos, can you see the results?

I do. I do. I see that most of the people do it, but not quite often. I’m sure you do have experience, but I hope that this webinar is helpful for you to develop your skills and be able to get the most out of your interviews with users.

Yeah, it looks like we do have some veterans on the call today. I would say a good majority doing moderated sessions, maybe once a quarter or less. And so again, our hope here today is really to give you some confidence when you’re doing your own moderated sessions. How do I prep? What do I do when I’m in the session and how do I analyze my sessions? Really again, give you some tactical strategies. So that one, you feel more confident. And two, that you just go out there and do more interviews with users. ‘Cause here at Userlytics, we believe in really the concept of user experience and you’re not gonna get those insights unless you actually get out there and do them.

So go ahead and stop the share on that poll and let’s jump into things today.

All right, let’s talk a little bit about our schedule. What’s our plan for the workshop today? We have about an hour. We are able to go a little bit longer if we have a lot of QA at the end. But first, just want to do a light introduction into moderation. Again, I’m sure that everyone on the call today is kind of familiar with what the technique is. But let’s talk a little bit about why it’s different and what are some of the different types of live interviews that you can do. And then we have Carlos, who will be breaking down again his best practices and strategies. Really a good time to lock in as he’s going through. Some advice and tactics that you can use. And then we’re actually gonna give you the opportunity to really put those to the test as we’ll be doing some breakout sessions. We’re gonna be breaking everyone into groups of about three and you’ll be able to practice with other attendees on the call and be able to try out some of these tactics. So that’ll be for a short amount of time and then we’ll jump in and we’ll give everyone a chance for some Q&A.

So I do wanna say, while we have the chat open and feel free to throw things in that chat, If you have a question for the Q&A and you don’t want to forget it, just go ahead and throw it in the Q&A section and we’ll cover it towards the end of the call.

All right, well, let’s go ahead and jump into the meat of things. I want to, again, talk a little bit about moderation from a high level and what kind of makes moderation different. Because again, there’s lots of different types of research that you can do. A lot of people are familiar with more quantitative types of user feedback. but moderation is really the opportunity to really understand your users in a deep, deep level. And a lot of things that you can’t do, and let’s say an unmoderated recorded session or some type of survey, is that you can’t change things on the fly, like you can in a live interview session. So as a moderator, we want you to be open to your participants to be able to dive deeper on maybe something that they mentioned in the session. You’re like, “Hey, I wonder why they did that.” to kind of pivot a little bit in a moderated session and really clarify what that participant meant.

Now, Carlos, would you have any maybe words of caution for moderators about pivoting from maybe their original moderation guide? What’s kind of the good balance there?

So as you mentioned, what I see is the best aspect of doing moderations is having the chance to do some changes on the fly. But I think that it’s really important and I think it’s the key for a good moderation and a successful interview to have a really well structured discussion guide. This is going to let you go through the flow in a structured way, but again, you have the to probe in certain areas, which is basically what we’re looking to have with these moderated sessions. Also, the fact that we can observe what users are doing and build questions in that sense, in that aspect, that maybe we can see in the moderated sessions, but we can not ask about those things. I think that’s the main thing. But again, it’s really important to have a structure and discussion guide and follow it through the test with your variations. Every interview is going to be different, but it’s really important to have that kind of guide for you, to know what to do, what to ask, and especially not to forget any of your research goals.

Couldn’t have put it better myself, thanks Carlos. And again, I’m sure we’ll dive into that a little bit more. And Brandon, appreciate that question. We will definitely get to it in the QA. And again, if you have more questions, please throw them in there, I’d love to see that.

But the other important thing we wanna, we wanna again focus on is building trust and report. ‘Cause this is something that you really get to do really in depth in a moderated session, which again, you might not be able to do in any kind of unmoderated scenario, is again, direct human interaction. So again, it’s important at the beginning of your moderated session to really focus on building that trust and report and not leaving that till later. So instead of just jumping in and starting to hammer your participant with questions and probes and almost having some sort of like an interview, like kind of like a job interview, you might say. Again, really focus on build a little bit of a relationship and that participant will open up to you. So Carlos again, will be sharing a little bit more on like how to build trust to report a little bit later.

And then last thing is blurbing those non-herbal cues. So this goes a little bit back to the real time interaction, right? We can see participants as we’re doing our live interview as maybe they’re going through one of our digital assets. And we can maybe see some things that are happening that we would not be able to see in a survey. We can see their facial expressions, maybe they’re showing discomfort, confusion, or maybe happiness. And again, those are really good things to look out for as a moderator is what’s happening on the nonverbal side. And we can kind of probe into that more. Maybe not too directly, but again, we can understand, Hey, they’re happy here or they’re frustrated here. Let me find a good question to help understand what that emotion is.

All right, couple of different interview types. Again, these are just different forms of research that you can do. So, moderated research is not just I simply jump on either an in-person or a remote session with a participant asking questions. We might wanna understand a little bit more what we’re trying to get out of this session and kind of build our moderation guide around this. So a few different ones that we’ll go through quickly.

And I want to just quickly pause and say that the recording and this slide deck will be provided to everyone who’s registered, whether they were able to attend or not. So just a quick tangent, we will be sharing the recording and the slide deck. So don’t feel like you have to make all of your notes perfect here today.

All right, so again, a couple of different interview types. Exploratory is one that I personally like a lot. It’s really not too structured. You get to really just have an open conversation with the participant. And this is really great for understanding the motivations, needs, pain points. And again, this is a great type of research to do early on. So let’s say we’re in a design process. We’re gonna make some changes to a website we built or a digital asset or an experience. And we’re not really sure what one needs to change. Exploratory, a great interview type for that type of area where we’re at in the design process.

Usability testing, again, these are really great for unmoderated types of research, but when we’re doing moderation, it is also great to include some usability testing in our moderation, again, presenting the participant with an asset, like maybe a prototype, and having them go through the experience, and we just get to sit back there and watch.

Carlos, quick question for you with usability testing. What is the balance between making sure we’re not leading the participant too much as they’re going through a prototype and maybe not sitting too much back and just letting the participant go through? Is there a balance there or something they should look at?

Yeah, so that’s a very good question, especially when we’re focusing on prototypes, the best way to avoid leading the user is actually defining what our journey is gonna be. I just, you know, disabling in the prototype, anything that is going to be, you know, background noise and just letting them do the specific things that we are going to test. That way we are not leading the user with her questions, but with their ability to actually navigate through the prototype. I see also Brandon asked a very good question about keeping a strict order with the discussion guide. important and it’s actually a big decision to take when creating our discussion guide. It all comes down to the goals that we are going to assess. So if we’re trying to get to understand if there’s a need for a product, that way I try to have it as open as possible, let the user bring up anything that comes through their mind. if we are testing, let’s say a website or an app, it’s better to have like a pretty fine order. Obviously, users can, you know, get out of track, but just make sure that they’re coming back so that you get your research objectives and your research questions answered with that with that type of organization.

Awesome. Two for one. Answer my question and Brandon’s. Love that Carlos.

A couple more interview types and then again we’ll jump into some of the things that Carlos had to share. So concept testing again this is going to be maybe in the earlier design stages. So while exploratory might be like our discovery phase. Once we do have an idea of what we want to change or what we want to build, and maybe we’ve started even building maybe some MVPs, concept testing allows us to really identify specific things, insights or actions that we want to proceed with later. So again, maybe we have a few different ideas on the team of how to build something or how to structure something in our design. And we want to know, hey, what’s the best pass? Should we take this action or that action? A concept test with a couple of participants really to just assess, you know, their perceptions and ideas that they have, again, can really clarify the right way to go. Instead of just, hey, we internally think we should go with route B, and then you go with route B, but you didn’t do any testing on the concepts, and now you might have to go back and make some big changes later. So again, we love moderation throughout the design process as best as possible.

And then the last one is focus groups. Carlos, I was really interested to hear your insights here. This is just instead of a one-on-one conversation with the participant, we have multiple participants. We usually recommend anywhere from four to eight, less than four. You may not get as good of a group discussion, but more than eight can be a little bit difficult to manage as a moderator. So Carlos is just interested in any advice you have for people who are interested in doing a focus group with multiple participants.

Yeah, so it’s very interesting because in focus group what we see usually is that there are two types of personalities, not strong personalities and those people who try to hold back their thoughts trying not to to start discussions. So in that sense, it’s really important to, you know, give time for everyone to express their opinion, also to try to build with the participants, which by the way, it’s not easy, uh, an environment in which they feel like they can express any of their thoughts without, you know, that being a source of conflict. So in that sense, it’s really, really important to, I don’t know, type the user answers to questions that you’re going to follow up, but maybe not with the same user, but follow up with a different user about what the first user said. That’s really a good practice. Also, focus groups have the challenge of time. There are some users that spend more time, so you have to really keep an eye on that. And also, some of the users that hold back you sometimes have to put a bit of more pressure on them in order to make them talk and to really get valuable insights from them as well.

Wonderful, thank you Carlos. So again, just to clarify right, focus group just means multiple participants in the interviews. So you know, in a regular moderated study, maybe an exploratory one, you might have four to eight participants, but the interviews themselves are one-on-one. So that’s kind of the difference with focus groups. But that doesn’t mean that you can’t do a usability test or maybe more exploratory research focus group. So it’s really just a different structure of the session itself.

All right, well that’s enough for me. Excited to learn from the expert himself, Carlos. Carlos just has again some best practices and tactics. This is a really good area for notes again, just to understand or have some good things that you can take post this workshop or even in the breakout sessions that we do. So I’m going to pass it over to you Carlos.

Thank you, Nate. So first thing that you have to do when trying to do a moderation is define what you’re trying to find. Now every research project must start by defining what we need to learn and in this case and for that we have to define clear but also realistic research objectives. We cannot be too ambitious, meaning that we cannot have 11 research objectives or research questions. We have to try to have around four or five, no more than that because those are probably going to stretch over 60 minutes, which is the the usual length that you should have on a moderated session. But the most important part is, you know, defining those also not being too big, because otherwise we’re not going to be able to create actual questions that are going to provide us with insightful answers.

So let’s say, for example, that we want to know how users behave in a purchase process in our website. So for that, we have to define what we want to learn. For example, we want to learn why or how do they think? What do they go back to? What do they revisit? Do they give a lot of importance to terms of conditions or to any aspect that maybe we just give for granted? So important not to make any assumption when defining these goals. Now that you can do some assumptions, but obviously this has to be backed by some research that you’ve done previously, but I’d say no assumptions at all. It’s the best thing to do while doing, while defining these research goals.

Also, it’s important to know who our users are. So we have to do some research or have some kind of information on who is using our platform. We maybe want to interview women that are looking to get pregnant or similar. And then we just have to make sure that these research objectives are according to that. Or for example, with accessibility studies, users that have some type of visual deficiency. It’s important to know that beforehand because that’s also part of the designing of the discussion guide.

The next part is creating an interview guide. So in this case, going back to what I said previously, you have to have this very well structured because you want to cover all your research questions. Obviously, once in the session, you can open the questions, you can try to get to certain areas, maybe at users say something that you didn’t think about and was interesting and you want to probe over on that. So that’s very good, but you know, you have to be mindful of your discussion guide to be able to answer all your research process. Questions, sorry.

So a bit of do’s and don’ts for this discussion guide design. I usually like to go from general to specific, asking all time open questions, but like focusing first on, okay, what would you do when you arrive on this page? And then going, you know, linking each question that go from general to specific, try to avoid yes and no questions, looking always for open and the questions, because that way you will We get the chance to allow users to elaborate on their answers and then we get the insight what’s the reasoning behind their answers.

Also I like also to define a clear starting point for the journey if we are doing some type of usability test, concept test, prototype testing. In those kind of stages it’s important to define a clear starting point. you want to start from the homepage, but maybe if you’re just testing the payment journey, you just want to start on the specific product page and see how they advance from that.

Also, it’s important to estimate a time for its specific section. As I said before, we have normally 60 minutes, but it’s important to understand that users tend to get tired as we advance. Normally, the user’s attention lasts for around 50 minutes or so. So we don’t want to actually reach to that point where they are exhausted. So for that, it’s really important to estimate a time for a specific section, for a specific question, so that we know if we’re going to be able to cover all of our discussion guide.

Also, some useful tips are using scales and quantitative questions to measure our KPIs, especially things like feelings, like impressions, etc. Which are really subjective. It’s useful after asking them what their impressions are or what their feelings were towards the page to, you know, introduce those skills in which they can rate how good or bad so that we have later those KPIs for the analysis.

And also, well, as mentioned before, some users do tend to get off track. And it’s important to include some prompts to keep them on track to, you know, try to take them to the right place. But again, and here’s where it’s difficult, not leading them, you know? So it’s difficult to get that balance.

And don’t, well, I’m sure you are familiar with all these, with the design of the discussion guides, but most important is to avoid leading the user too much. Avoid threatening questions. If you want truthful answers, there are certain topics that users tend to lie or feel uncomfortable answering some things like what’s your income level or what’s your sexual orientation. Those are some type of questions that users tend not to be comfortable answering. So it’s important to avoid those threatening questions or at least formulate it in a way that they don’t feel threatened.

As said before, avoiding yes and no questions.

And very important is to give users space. Don’t overload the user with questions because otherwise they’re not gonna have enough time to explain their reasoning behind their answers. And you’re going to be losing important insights back there.

Also, it’s important to measure time. And if you can do a try run with some real participants, if you can, and if not with someone that has not been involved in the design of this discussion guide. is basically going to give you enough information about the time that you are taking for each questions and it’s going to let you know which, if you have to cut off and which sections you have to cut off.

As mentioned, user’s attention lasts for around 50 minutes, so your discussion guide should be wrapped up in 45 minutes.

Also, you have to plan for the unexpected, you know, maybe the user arrives late or maybe you’ve spent too much time in one section. Well, that’s why we have that buffer of time.

And lastly, ensuring technical and logistical readiness. So here, for example, I want to link with the question that’s already brought up. What’s the best it set up for a moderation interview? Well, there are multiple tools for remote testing. That’s something that you have, like these have to be reliable. You have Zoom, which is kind of the most common, but Userlytics, for example, offers, apart from the moderation services, some type of tools later for easing the analysis, and that can make really a difference when doing that analysis.

And also the recruitment and scheduling. You have to do recruitment and you have to do scheduling. You normally, I recommend over recruiting because there tends to be no shows. So, you know, opening, if you have 10 users, opening it to 12 and looking for 12 users and scheduling, making sure that they’re gonna show up and sending them reminders. Hey, you have a session upcoming in two hours. Remember that and so on.

So if you could go to the next slide, please, Nate.

Yeah, sure. I just want to hit on one point, ‘cause I think it’s important, right? ‘Cause there are a lot of tools out there, things like Zoom, and things like Zoom and Teams will require you to have to do a lot of the logistic steps yourself. Obviously something like Userlytics, automates a lot of that for you and makes that super, super easy. Also, lots of great questions in the Q&A. Keep those coming. We’re gonna answer those afterwards. And real quick, we want to hear from you guys. If you’re getting useful tips so far, or you feel like this has been useful, put a thumbs up in the chat. We wanna know if this is going well so far. All right, back over to you guys.

Yeah, so, well, the interview, this is the actual meet, let’s say, of the research process. So here, the best thing that you can do, or at least what I do when preparing the interviews, finding a quiet place, a place that I’m on my own. I have no distractions because I want to give the user all my attention. This is really important because users have to feel like you are paying attention to them so that they open up to you and give you their reasons, give you their logic and also that they are truthful with their responses. So just make sure that you are giving all your attention to the participants.

Some researchers tend to do some note-taking while doing the interview. I am not so fond of that because when you start typing, you probably miss out something or they don’t see some action that the user does on the website. So I prefer doing that afterwards, but obviously there’s people who are better than me and multitasking. So if you feel comfortable, that’s it, but make sure that you’re giving all your attention to the participant.

The next part, and this is just at the beginning of the interview, you have to build a connection with the participant. You should introduce yourself, tell them a bit about yourself. And here’s where a really small talk comes in handy. You know, if you’re doing some tests for a school website, maybe you’re looking for parents, well, you can ask them about their kids, how old are they? What curves are they in? Are they boys, girls, something like that so that they feel like you care about what they’re telling you and they’re more likely to open up.

And also instructions. This is really important. You want to have the user to be comfortable so that they don’t have to feel alert by anything. So in that sense, it’s important to explain them what we’re going to be doing, what we’re looking for specifically, obviously without giving out all our research questions, but we have to tell them, okay, so today we’re going to focus on the aesthetics of this page. So, you know, I want to get your feedback on what do you think of the layout of the design, etc. Explain the instructions, explain how much time you’re going to be with them, explain to them that they are no right or wrong answers. This is really important. And basically telling them what we’re going to do and for how long.

Also, once you start with the actual test. It’s what we call active listening. So listening to the user but making them feel like they are listening. So asking them questions, probing on certain areas, seeing where they are spending more time, all that is going to give us hints on what’s important for the user and as tips that are not so much of a useful conversation that you will have with someone in your house, you have to have this visual, what I call visual listening. So for example, where does the user keep going back to? Do they keep going back to the product page? Why are they going back to the product page? Is there anything that was not clear? Is there anything that they forgot about? So, you know, those hints let you probe on certain areas and get more information and especially getting to know what the thinking process of the user is.

There are also other hints like mumbles or facial expressions, as Nate told before, and this is also going to give us some hints of something that maybe they are not understanding or maybe they are surprised about it. So basically getting to read all those hints and get to know where are our issues.

Also silences. Silences are very important. When users spend, let’s say, I don’t know, 15 seconds in silence, it’s probably because they are thinking about something because they’re reading something, because they are focusing on something. So don’t try to rush and, you know, fill the void. On the other hand, you have to take your time, don’t probe, and also give them the space to rethink. So if immediately after they answer your question, you reply with another question, you’re not giving them time to think and maybe they can give you some more insight. So I would say that just keep a slow pace while moderating so that users can have the time to rethink and probably give you some more insights.

And well, obviously some politeness is necessary. So, you know, don’t cut off users, don’t make assumptions. And that’s that way you would get most out of them.

And also just, you know, like the golden rule, I would say is ask users, but without questioning them. So when they are giving you an answer and you want to know why, you don’t want to, you know, that why to feel like, why do you think that? you’re wrong, you want to, you know, okay, can you tell me a bit more about that? Is there anything that has made you feel that way or that has raised that thought? What would you do next? Why would you do that? Something like that. So, you know, don’t question the user because that way they would probably close up and be less truthful and you’re probably going to be losing very important incentives.

And also, well, some things that I would also like to do, one possible is to have, you know, hidden observers, team members and other people involved in the research so that they can see some things that maybe we miss. Maybe they have ideas on where to probe that maybe we don’t come up with them because we’re talking with the user. So having someone to assist us in that process is also very, very interesting.

And finally, well, managing time effectively, as mentioned before, you have to have, you know, schedule how much time you’re going to spend in each section. If you have, let’s say, 10 users, you don’t have to cover everything with all the 10 users. If you are running out of time with one user, you can jump one section and you will cover it with another user. But you know, make sure that you have your time in mind, most tools like the Userlytics one have a timer and have reminders like you have five minutes left. So try to not rush the user and not overshadow it. And if you have to skip one section with one user, that’s completely fine. You have another nine users that are going to be going over that section with you.

And finally, just stay neutral and avoid leading questions. As mentioned, this is obviously more important when designing the, this is something you’re gonna put in practice when designing the discussion guide, but it’s really important to keep it that way when you are on the interview because that way, the user doesn’t feel led or doesn’t feel like you want them to answer one thing. So try to keep neutrality, obviously, but you have to still feel human, not you can not be a robot. So I don’t know if you guys have any questions here, but for the last part, basically, summarizing the insights.

Normally what I like to do after doing an interview is spend like 15-20 minutes wrapping up, not taking what were the key points of the session and highlight the most relevant aspect and also identify what are the most severe issues.

In Userlytics, for example, we referenced the Nielsen-Norman group as, I guess, it’s a great reference in the UX research industry and their three criteria for determining how severe an issue is, is basically frequency. How often does this happen across users? Is it something that just one user came across or all of the users came across this? Also, the impact, is it something that they can overcome or is it something that just ends the year in there? And lastly, the persistence. So do users keep coming across it, or is it something that just happens in one point? Do they come across several times during the session, or is it something that just, it’s a one-time thing?

And well, as I just mentioned before, collaborating with hidden observers is going to, let us do focus on the interview, what they do the note taking, but also give us some insights on how, on where to probe and how to continue the interview.

And finally, sharing the insights with your stakeholders. So here, the most important thing to think about is who am I sharing this with? Is it going to be the CEO or is it going to be another researcher? This is mostly the most important decision. And here, there are multiple formats like PowerPoint, video, text, documents, et cetera. And in Useryitics, we have a very interesting tool that I really personally have used a lot and the clients really like is the Highlight Reels, which basically you can bundle a lot of clips from different users coming across different issues so that they don’t have to see the 60 minutes of the sessions, but you just select the most important points and that way they can see themselves and can get some of their conclusions.

Wonderful Carlos. And again, shameless plug, Userlytics is a tool that would really help in this process. Again, being able to share out video clips, being able to identify points in the session instead of having to try to remember them post session. But Userlytics is maybe not necessarily the only tool that can do that, but it’s one that we obviously encourage here.

All right, thanks Carlos. Again, some really good questions. We will definitely hit those questions towards the end, but we want to again, as part of this workshop give you all a chance to really put some of these strategies to the test and we’re going to do a little bit of a breakout session. So just a quick overview of like what that means and what we’re going to be doing.

So how this breakout is going to work is we’re going to be splitting everyone into groups of about three. Now again, someone may drop off and you may only have two people. That’s totally fine. The intention is to have one moderator, one participant, and ideally one observer as part of the breakout sessions. And then we’re going to rotate those roles every three minutes. And I’ll be sending messages to the groups again just about every three minutes so that everyone knows when to switch.

And again, just a couple rules of the road here. You know, be gentle with everyone, right? Some people have more experience than others and this is again no pressure at all just an opportunity for us to all practice a little bit get some feedback maybe ask some questions to each other as we go through so I’m going to be creating those break rooms right now go ahead and assign those and get us in the group here.

All right all right I think we have a timer everyone joining back in breakout rooms finished. If there was any trouble, I was moving some people around because some people dropped and some people were alone, so I was trying to combine groups. I hope that went smoothly for everyone. Carlos, what did you see in the breakout room? Anything good?

Well, I did. I jumped in. I listened to some of the moderators. I think that they did really good. I also got some feedback and they gave some feedback about their moderators. But yeah, most important, I was asking two of the participants, Katherine and Katia, about if they actually got the research questions answered, which is basically what we were trying to do here. So, getting the participant to feel comfortable, but at the same time, having our questions answered.

I love it. And, uh, Shiva, I did see that you weren’t able to get in. So I apologize. Um, after the session, more than happy to connect with you and get a little practice. You and me, uh, want to give you a chance to do that. So I’ll connect with you afterwards. Um, but in the chat, I see some, um, some good notes going in there. We’re interested to hear from you guys. So we’ll spend maybe a couple minutes before the Q&A. Tell us in the chat, what was one piece of maybe good feedback you got from a group member, maybe a strategy that we covered that was useful in the breakout simulation, or any maybe difficulties, not technical, but any difficulties that you found when you were doing your own moderation. Throw them in the chat, we’ll read them out as they come. Again, we’re just interested to hear back from you guys.

And so we’ll, again, it looks like some thanks, some apologies, all good. But now we wanna open for Q&A and then there’s a bunch of good Q&A questions already and so we’re gonna answer those first. Again, Carlos, you’re the mastermind, so I’ll be asking these to you if I can provide any input, I will. But again, if you have any questions, maybe in the breakout simulation, you’re like, hey, how do I handle this? Throw them in the Q&A, but let’s go ahead and dive in on those. The first one I see from Casey, Carlos, how do you decide between using an Isodality or a low-fidelity prototypes in specifically usability? That’s a good question.

Yeah, that’s a very good question. So once, basically, this is going to be decided on whether what your research objectives are. I know that maybe it’s a bit repetitive of me coming back to this, but that’s mostly what’s going to define, basically, the methodology and what type of questions you’re going to ask. If you are trying to test a single journey within that platform, then a low-file prototype is probably going to be better. If you want to test the overall journey and you want to understand where users want to go, you have to have everything or most of it accessible, meaning that probably a high-fidelity prototype is going to be something that you’re going to be benefiting more of.

Love that and appreciate that question.

Next one is a little in-depth. I’m going to do my best to read this off from an anonymous user. In usability testing, should I have users complete the task first and then discuss further details on the web page or guide them step by step while asking about specific web page sections? A good question over to you, Carlos.

Well, yeah, this is something that is really interesting. This is, I usually tend to give them, especially at the start. I want to learn what they want to do, what they come to do to this website. So I just tell them a bit of what’s this website about and have them go through the website. Maybe I put them somewhere or two gold, one or two golds, like, you know, find where these specific companies located in real life or something like that. And then after I’ve seen how they behave in the website, try to go back to those questions. Obviously this depends on the time and the amount of pages that we have on the website. So on that sense, if you feel like you have too much and users are probably going to go get off track, it’s better to be guiding them, obviously not step by step, but yes, guiding them, not leading them, which it’s hard to maintain that balance.

Totally hear that, I love that. I do wanna be cognizant. We’re about to run out of time here, but we are going to stay on a little bit longer, up to 30 minutes, just again, to get all these questions answered. So if you need to drop off, totally understand. We will still answer the questions live. It’ll be recorded. Recordings will be sent to you. So if you have to jump, we appreciate you spending some time with us and sticking through the breakout sessions. Let’s keep going on these questions, Carlos. The next one is from Brandon. Is it a good practice to share the interview guide with questions with, I believe it’s overseers or maybe observers, is there too much info to share?

So I usually tend to have some researchers, not of my team, but of the client sides, join as observers. Usually I work with them on the creation of the discussion guide. So in that sense, they do have access and that’s good, but I think it’s not something that’s key. It’s up to you know, if there’s any type of privacy issues or whatever, but for the most part I would say that you would benefit more if they have the discussion guide because you know they’re not going to be throwing you follow up questions or pro questions that are already on the discussion guide and you’re going to be able to you know have the focus and that way they can you know do some more insightful, some more insightful questions you so that you can raise it up to the client, to the participant’s story.

I love that, Carlos. This one is actually really interesting. I saw this one earlier in the workshop and I decided to hear your thoughts on this Carlos. How to act or maybe some, you know, some of your thoughts Carlos, of how to act when participants have emotional moments, right? Maybe for example, they have a moment where they’re crying or they’re a little bit angry, right? They get a little emotional. How to– so I guess that would be the first question. The second part would be how to avoid making judgments and be able to bring them back on topic.

Well, it’s actually very interesting, because you actually see when you have so many moderations, as I’ve done, see people that– well, not crying, but getting angry. It’s something that usually happens. And in that sense, obviously you’re not there to try to calm them down or anything. It’s the website that is frustrating them. So it’s up to the website calming them down. So in that sense, it’s more of getting some information on why they got angry and maybe what can be done to solve that. And how to avoid making judgments here. I try to put aside every, like all my convictions, all my opinions and just, you know, try to, you know, say yes or agree mostly with all of the use of what the user is feeling so that they feel like we’re on the same page and that way they don’t feel judged and again they provide me with truthful answers and also you’re not going to start a discussion with the participant on a live session.

Yeah, I love that question. I’m interested in the chat. If anyone of the attendees, right, if you have experienced someone getting emotional, what was that experience for that? I’d love to see that in the chat. Carlos, let’s keep going on here. Next one is from Casey again. “Do you have any tips on impromptu speaking? When you want to maybe probe something, a participant brings up that you didn’t think of beforehand. Maybe tips and tricks, again, not to like cut off the participant. How should you kind of maybe engage in some impromptu moments?

Yeah, so that’s really that’s a really good question. I tend to, you know, just let them finish talking and instead of, you know, going through the next question, just ask them about like, hey, so you were up something that actually caught my attention. And, you know, I want to understand a bit more about that. So, you know, that kind of language, just, you know, circling back also referencing what they say is really important so that they feel listened. So I think that’s really good. And, you know, I feel like here it’s not so much about the prompt or the questions that you raised, but it’s more about, you know, having the ability to read those type of things. So it’s really, it’s really just, you know, say mentioning them, something that they brought up that caught your attention or that you want to learn about more. That’s kind of what I use for, you know, gathering information over there.

Yep, I love that. Good, good note from Laura in the chat. Thanks, Laura.

Next question from Mike, who works in financial services. We’re actually going to have a couple questions here about kind of finance healthcare. But Mike’s question, I work in financial services and one thing that has been consistently difficult is broaching these uncomfortable topics like income. Do you have any insights and had a handle? Right? Sometimes people get a little hesitant to share personal information, how do you kind of broach that topic, Carlos?

So in that specific aspect with those threatening topics, it’s something that there’s not really much we can do about. We can reformulate the question so that they don’t like feel that way so exposed. But even that not always works. I think that on that side, the best thing that can be done is when doing the recruitment part, making sure that the participants are comfortable, we’re sharing that kind of information saying like, hey, this session is going to be asking about things like your income or your sexual orientation or anything like that. Are you comfortable discussing that on a live recorded session? If not, I should just try to go. You are going to find users that are going to be comfortable with that. The thing there is just you will probably find the bias because it’s going to be the best well of users, the ones who won’t have any problem sharing that rather than those who have more struggles. So it’s just kind of a, it’s a hard difficulty to overcome, but I’d say that reformulating questions in a way that they don’t feel that that threatened, it’s going to be your best chance. And go around like, do you do vacationing abroad or do you do vacationing in within your state? So those kind of things are going to give you hints or where are they standing? But obviously, you’re not going to have like a specific number that that’s going to give you.

Yeah, and I think maybe from a logistical standpoint, again, consent from the participant is important and it kind of also ties into Laura’s question, which we’ll go through next, right? How do I get the consent and make sure that the participant is willing to kind of go through those topics and share that information before we get on the call? And again, using something like Userlytics allows you to build those into your screeners. So we’re able to ask participants before any call or anything. They’re able to kind of see what may be sensitive subjects we’re going to speak about in the moderated session. And they’re able to kind of consent to that before jumping it. So a good question there. And I guess this next one kind of ties in Laura’s question. Any recommendations, Carlos, for UX research, where we want to get feedback on digital tools using finance or healthcare industries. She’s curious to know if there are ways to get around privacy concerns with PII, personal, identifiable information, while still seeing actual behaviors. And Mike actually commented seconding that. He said, “We often reach a limit of usefulness with fake data.” Thoughts here, Carlos. I think this is an interesting topic.

Yes, this is very interesting. As Mike mentions, it’s hard to, like you get up to some point where you just, using fake data is not going to give you that information. There are some tools that allow you to black out the screen of the user and just, you don’t see the interaction. That way you don’t see that privacy concern and the, sorry, PII, but at the same time, you lose that chance to see how they interact. So what we usually do is just either provide them with fake data and just try to get the most out of it. It’s a really interesting topic, but so far in the industry, there’s not really a consensus on how to get over this. And this is something that it’s limiting. And I’m sure that it’s going to be a challenge for the upcoming years as more and more financial and healthcare tools get developed and users want to to ensure that these are usable and user-friendly for them.

Yeah, and again, I would just reiterate that sometimes at Userlytics, what we do in these situations, again, we’ll preempt the participant, right? So they’re only allowed to book a session and actually join the session if they agree that, you know, they’re gonna be sharing this personal information. So it depends a little bit on what the information is, right? like an email is very different from like social security number, things like that. So always good. And again, if you need help, kind of how do I approach this? Newsletters is here to help.

Last question from the Q&A, Carlos. And then we’ll wrap it up here. Is smiling or nodding on participants’ answers, does that act as a kind of bias in an interview moderation, a really cool question?

Yes, very cool. So I would say it’s actually the opposite. Obviously, if a user says something seriously you laugh at it, maybe that way they are going to be feeling like you know you are making fun of them. So obviously not, but you know when they, there are many users, especially experienced users that you know like doing some jokes or you know commenting in some areas. So you know in that sense I would say that it’s more important to be in empathy with them than staying like a robot because that way they’re going to be like okay so this person wants to find out this this this and they’re not really going to elaborate on their reasoning. So I think like, you know, nodding especially makes them feel like you are listening to them. And, and against, you know, if they tell a joke, you can laugh. It’s not like you’re here on a test for yourself. You want to build that connection with the, with the user and, and, you know, just you have to go with them and make them feel like they are funny.

Hello, back Carlos. Well, we are over time here and that was all the Q&A questions we had, so I’m going to kind of skip most of this last part, but just wanted everyone to be aware that usual UX can do the moderation for you. So if you’d like someone like Carlos to actually moderate your sessions and maybe do a couple where you sit in as an observer and kind of learn from him, seeing him do it live or one of our other researchers, we offer this as a service that we would love for you to try out if you’re interested. We speak a lot of different languages on the the consulting team.

And then as a kind of bonus, so for anyone who registered and attended the webinar, we do actually have a special offer that through the month of February, 20% off on any UXC service, UXC just stands for UX Consultant, any moderation related services, 20% off. So if you’re already a part of Userlytics and using our tool, just reach out to your account manager to kind of, I would just say, take that offer. And if you’re new to Userlytics, but interested in having us do some for you, we’ll be again following up after.

Carlos, I think we had one last question before we wrap it up for sure here from Vanessa, who since she was saying, I stay with us, we’re going to answer this question. Carlos, have you, have you all utilized any AI tools to act as a hidden observer rather than using a person’s time on your team. So this is a good question. So I would actually, I’ll take this one Carlos, because we do have a feature within Userlytics that is called AI UX Analysis that will actually review your transcripts of any recorded session. So it could be unmoderated or moderated and will actually give you insights that it finds. So it’s AI built into the Userlytics tool. If you have the tool now, you should have access to it or we can turn that feature on but that is a way so the AI is not necessarily sitting in the session with you, but it’s analyzing the transcription post. So it’ll do that for you, give you a really nice output, things to look for. Appreciate that question, Vanessa.

Carlos, really appreciate you spending the time with us today. Really thought that what you had was insightful. Again, for everyone who’s still on, we thank you for spending a little more time with us today. Hope you’re having a great week so far. We’re going to be sending these workshop slides and the reporting afterwards. So don’t worry, automatically sent to you. And we hope that you not only spend some time with us, maybe even utilize our consulting team. But with that is all we have for you today. Any last thoughts, Carlos?

No, no, no, I hope that this was helpful for all the researchers that came in today. And well, for anyone that decided to sit in with us, I saw very interesting questions and very good moderations on the back end. So really good. And yeah, thank you very much for having me, Nate.

All right, we’ll take care. Everyone will be chatting with everyone and we’ll be back with you soon. Recording and slides coming up next. And thank you for your time. Have a good one.

Meet Our Experts:

We are excited to have two distinguished speakers joining us to share their expertise:

Nate Brown

Nate is an accomplished account manager for many large enterprise-level companies in the North American region. With multiple years of experience collaborating with research teams to maximize their research in the Userlytics platform, Nate possesses key insights into why some research projects lack substance and others produce valuable insights. His favorite part about working at Userlytics is building lasting relationships with his clients, even in a remote setting.

Schedule a Free Demo

Carlos del Rio

With a degree in Sociology and Political Science, Carlos brings a strong foundation in social research, specializing in online and quantitative methodologies. With over three years of experience as a UX Researcher, including a senior role, he excels in leading user-centered projects.

Schedule a Free Demo