One Quick Lesson About (Over) Relying on Data - Build with Maggie Crowley Recap

Podcast: Build with Maggie Crowley

Published: 2021-04-09

Duration: 6 min

Summary

Maggie Crowley discusses the importance of balancing data analysis with qualitative customer feedback and personal product use to ensure a comprehensive understanding of product performance.

What Happened

Maggie Crowley recounts her experience managing a product launch where she extensively relied on data dashboards to assess performance. Initially, the product seemed successful based on quantitative metrics, but customer feedback revealed dissatisfaction with the user experience. The disconnect between data and customer sentiment highlighted the need for qualitative insights, prompting a deeper investigation into customer experiences through conversations and reviewing product interactions.

Maggie emphasizes the importance of pairing data with qualitative research to get the full picture of product performance. She advises against over-reliance on data alone, as it can mask underlying issues that affect customer satisfaction.

To address the gap between data and customer experience, Maggie and her team began conducting more user interviews and analyzing customer interactions with the product. This approach provided valuable insights that data alone could not offer.

Maggie stresses the importance of product managers using their own products to understand user experiences better. She notes that managers, in particular, risk losing touch with the current state of the product if they focus too much on future developments.

She advises asking critical questions about product performance, customer satisfaction, and personal interaction with the product to identify areas for improvement. These questions help ensure that data analysis is complemented by firsthand product usage and direct customer feedback.

Maggie shares practical tips for product managers, including using SQL for data analysis, conducting unbiased user interviews, and dogfooding their own products. These strategies help create a balanced approach to product assessment and improvement.

The episode serves as a reminder that while data is essential, it should not overshadow the qualitative aspects of product management. By integrating both quantitative and qualitative methods, product managers can gain a more accurate understanding of their products and make informed decisions.

Key Insights

• Relying solely on quantitative data can lead to a misleading assessment of product success, as seen in cases where positive metrics mask underlying customer dissatisfaction.
• Integrating qualitative research methods, such as user interviews and direct product interaction analysis, provides insights that data alone may not capture, helping to uncover user experience issues.
• Product managers are advised to regularly use their own products, a practice known as 'dogfooding,' to maintain a direct understanding of the user experience and identify potential areas for improvement.
• Practical strategies for balanced product assessment include using SQL for data analysis, conducting unbiased user interviews, and actively engaging with customer feedback to complement quantitative data.