10X FASTER
Turning 10 to 20 hours of manual analysis into 1 to 2 hours with AI Insights.
Without a dedicated UX researcher, ArmadaCare’s VP of Marketing successfully ran a full usability study with support from Userlytics’ UX Consulting team, and analyzed results 10X faster with AI Insights and using Highlight Reels to keep leadership in the loop.
At a Glance
| Company | ArmadaCare |
| Industry | Supplemental health insurance solutions for brokers, partners, and employers |
| Contact | Michelle Johnson, VP of Marketing (Brand) |
| Study length | 2 weeks |
| Research team | Userlytics’ UX Consulting team on behalf of ArmadaCare and Johnson. |
| Tools used | AI Insights, Highlight Reels, AI Annotations, expert moderation by Userlytics |
| Key result | Confident, record-time analysis that kept the member portal build moving forward. |
“AI in UX is not taking the place of researchers, it’s a time-saving tool. Used the right way, it can be a very powerful business tool, and not detrimental to teams.”
Michelle Johnson, VP of Marketing (Brand), ArmadaCare
The Challenge
When an outside vendor couldn’t keep pace with the development of a new Member Portal, ArmadaCare’s team took development and research into its own hands.
ArmadaCare, which brings supplemental health insurance solutions to brokers, partners, and employers built around its Ultimate Health® executive medical reimbursement plan, had been partnering with an outside vendor to build a new member portal and mobile app. When that vendor couldn’t give the project the time and attention it needed, ArmadaCare decided to build it in-house.
That decision put the research on Michelle Johnson’s desk. As VP of Marketing within Brand, she had no dedicated UX researcher on her team. She needed a partner who could set up and moderate a proper usability study, and a way to make sense of two weeks of results herself while keeping leadership informed throughout. It wasn’t her first time working with Userlytics either. She’d used the platform once before, for a website project, and came back for the same reason: an outside, unbiased read on whether a new design actually works for members, not just for the team that built it.
“We talk a lot internally about the dangers of drinking your own Kool-Aid. We don’t want to get to a point where we’re blind to what the journey is for users. Having those gut checks, having that outside view, it’s really important to make sure you’re adapting to all types of users.”
— Michelle Johnson, VP of Marketing (Brand), ArmadaCare
Solution
A hands-on moderator, and AI-powered tools such as AI Insights, AI Annotations and Highlight Reels, turned one person into a full research team.
Johnson set up a two-week study testing a Figma prototype of the new portal with real ArmadaCare members. With no dedicated UX researcher on her team, she brought in Userlytics’ UX Consulting team to set up and moderate every session, while she sat in on each one herself.
While her moderator, Tiago Moresco, one of Userlytics’ Principal UX Researchers, was well-briefed about the company, he wasn’t necessarily an ArmadaCare product expert, so Johnson was able to provide him real-time context through the internal chat feature whenever a member said something that needed translating. “It was like he was the expert in the room,” she says, “even though he’s not as familiar with our product as I am.”
During each session, Johnson used the annotations feature and tagged key moments in real time. She had skipped the feature for her first couple of interviews, then picked it up once she saw the payoff.
“I didn’t use the annotations feature in the first couple of interviews. And then I did, and I was so glad I did, because it enabled me to go back and find those true hidden gems within a 30-minute session.” — Michelle Johnson
Those annotations, alongside the platform’s AI-powered annotations, provided key insights that would later be analyzed by the Userlytics user experience AI assistant.
Once the studies were completed, Johnson was adamant about reviewing and analyzing the results herself. With two weeks of session data for Johnson to make sense of on her own, she tapped into AI Insights, an AI-powered user experience analysis workspace launched earlier this year.
AI Insights, Explained
AI Insights is Userlytics’ AI-powered user experience analysis feature. It’s built to guide UX researchers and non-researchers from raw sessions to insights, fast.
- Instant global summary: the moment a study wraps, AI Insights generates a summary of every session, providing the key highlights.
- Ask anything: follow-up questions get answered in natural language, pulled directly from the session data.
- Built to iterate: if an answer isn’t quite right, rephrasing the question gets you there.
- One document, ready to share: findings are compiled into a single report that’s easy to edit and hand off to leadership.
Johnson isn’t a trained UX researcher, but she found the tool intuitive from the start. It’s a process that “comes naturally with the tool, very straightforward,” she says. “I just ask questions and rephrase as needed until I have the answers I need.”
What made her trust the output was that she could check it against what she’d witnessed herself during the live sessions. For example, when AI Insights flagged one data point to treat carefully, tied to a member who’d admitted on camera she wasn’t familiar with the technology, it matched exactly what Johnson had seen live: “you can ignore that momentary confusion, because that had nothing to do with our results for the actual portal we were testing.”
It also worked mid-study, not just at the end. When her boss asked for a status update partway through testing, she had an answer on the spot instead of digging through handwritten notes: “I knew, because every single tester said it, I could tell him off the bat. But then I could also deliver a report quickly using AI.”
Johnson also needed to keep leadership current throughout the study, including her COO, who was out of office for the entire two weeks and still expected a full report the moment she was back. Rather than send a written recap, Johnson used the Highlight Reels feature, short auto-compiled video clips of key session moments, paired with a quick reference naming which member appeared in each clip and a link to Salesforce for anyone she wanted more context on.
“I showed her the Highlight Reels capability, and she said, ‘That is perfect. I would definitely take the time to go and watch a two, three minute video.’”
— Michelle Johnson, on her COO’s reaction to Highlight Reels
Results
10X faster analysis, without losing confidence in a single finding.
Consistent feedback across sessions surfaced early enough for the team to pause development and adjust before building further. ArmadaCare picked testing back up once new sections of the prototype were ready, instead of starting a new engagement. “I reached out to my Account Manager and he confirmed that my BYOU credits lasted for so long. I really appreciate that in a partner,” shares Johnson.
The biggest shift was time. Johnson estimated that distilling two weeks of sessions by hand, watching recordings, cross-referencing notes, and building a report herself, would have taken 10 to 20 hours. With AI Insights, the same work took 1 to 2 hours: 10X faster, without losing confidence in the results.
“We don’t have a dedicated researcher in-house, so distilling a two-week study used to mean giving up a weekend. With AI Insights, what would have taken me 10 to 20 hours took an hour or two, and I felt confident in it.”
— Michelle Johnson, VP of Marketing (Brand), ArmadaCare
Beyond the time saved, Johnson expressed that the partnership let her focus on what she valued most: connecting directly with ArmadaCare’s members instead of managing the mechanics of UX research.
“Userlytics does such a great job running studies on behalf of clients that I don’t have to worry. I can really sit back and focus on learning. It reinforces the importance of our transformation and why we’re spending all this time and energy on making sure we build things right.”
— Michelle Johnson, VP of Marketing (Brand), ArmadaCare
Michelle’s Tips for Running Research Without a Dedicated Team
- Sit in on your own sessions. Johnson attended every interview herself, which gave her the context to know what the data actually meant, not just what it said.
- Ask AI Insights plain questions, and rephrase if needed. “Some of it is trial and error. If you don’t get the type of answer you’re looking for, go at it another way.”
- Verify what AI tells you against what you’ve seen. She caught cases where a flagged data point traced back to one tester’s unfamiliarity with the tech, not a real product issue, because she’d watched the session herself.
- Remember it’s a tool, not a replacement. “AI in UX is not taking the place of researchers, it’s a time-saving tool. Used the right way, it can be a very powerful business tool.”
Ready to move 10X faster on your next UX study?
See what AI Insights and Highlight Reels can do for your team.
