Product Decisions Using Qualitative Data by Redfin Product Leader, Temi Moju-Igbene

What is qualitative data?

Qualitative data is narrative and descriptive data that gives you personal, yet subjective insight into your customers.

It helps you understand the why behind users behavior.

Examples of Qual data?

• UX research methods:

  • User interviews

  • Open ended survey questions

  • Usability studies

  • Focus groups etc.

• Support tickets

• In - app feedback

• Ideas portals

• Customer Reviews

Buckets of Qualitative Data

• Generative Research

  • helps you identify new opportunities e.g user interviews

• Evaluative Research

  • helps you assess/evaluate an existing feature or anexisting problem e.g. usability testing.

• Customer Driven Data

  • Mostly "unsolicited" customer feedback e.g support tickets.

Conducting Interviews and User Tests

Let`s Practice: Interviews

Research Question(s)

How do employees work on the go and what key capabilities are necessary to keep them productive

Example Questions

• Intro

  • What does a day of work look like for you?

  • Do you conduct work on your phone? When?

• Main

  • What kinds of tasks do you complete on the go?

  • How successful are you at completing said tasks?

• Conclusion/Wrap up

• If you could only do one work task on your phone, what would it be?

Let`s Practice: Usability Testing

• Initial

  Impressions/Feelings

  • What do you think when you look at this page

  • What are your initial impressions of this page

• Discoverability

  • What do you think you can do on this page?

• Testing Copy

  • What do you think would happen if you click on this button.

Analyzing your Results

User Interviews

• Ideally transcribe interviews

• Group insights and answers into themes

• Keep grouping to you have key takeaways

Usability Testing

• Note both success and failure

• Unnoticed elements

• Word and action mismatches

• Common mistakes as well as outliers

"What is the value of qualitative data?"

Its power is really in helping you understand what to do, especially when you use it in combination with quantitative data, because it takes a lot of resources to create something new. It actually takes a lot of resources to build something that already exists. Take this time to understand the space in the client, know who you are targeting, and understand their behavior patterns. This gives you the ability to make informed decisions and deliver what people want.

The value of qual data?

• To provide additional context for quantitative data findings.

• To identify a new product area or direction.

• To validate or invalidate feature ideas.

• To understand how/if an existing feature addresses a user goal.

Mistakes to Avoid

• Don`t try to quantify qualitative data

• Make your product directive insights not human directive insights

  • Underrepresented backgrounds get most affected by this.

  • If differs from your regular pool. It shows you need more insight.

• Qual is subjective so not great for preference testing.

• Try to get a diverse pool of users.

  • Don`t make population level generalizations.

Key Takeaways

• Qualitative data gives you insight into the why of customer behavior.

• Generative, evaluative, customer driven.

• User interviews and usability testing are two key research methods for data collection. For both preparation is key to data accuracy!

• Qual data goes hand in hand with quantitative data.

• Don`t try to quantify your results and always focus on product insights not human insights.

Temi Moju-Igbene, Redfin Product Leader