Why Quantitative Analytics Is Not Enough To Know Your Customers

Share the knowledge

Google Analytics, MixPanel and Kissmetrics are great product management tools to help you plan your roadmap. They give you powerful analytics and insights into what your customers are doing in your application. You can rally your team behind the data you get from these tools. You can analyze your week-over-week results to gauge how well your product is doing.

But, these product management analytics tools are not enough. Here are a few reasons why.

Known Unknowns VS. Unknown Unknowns

There are four kinds of information:

  1. Things we know we know. These are facts.
  2. Things we know we don’t know. These are questions we can answer by reporting.
  3. Things we don’t know we know. These are intuitions, which we should qualify and teach to improve effectiveness and efficiency.
  4. Things we don’t know we don’t know. These are exploration, which is where unfair epiphanies live.

Some examples of known unknowns are how many users you have or the number of lines of codes. Analytics tools can help you find out your “known unknowns.” You can use metrics for accounting (“how many widgets did you sell today?”) or to measure the outcome of an experiment (“Did the green or red widget sell more?”).

The “unknown unknowns” help you discover something new. This is where the magic lives. It can give you new information to help you disrupt a market and lead you towards a “eureka!” moment. Analytics alone won’t help you discover your “unknown unknowns.” This is where qualitative data from focused conversations with potential customers in the right market can help you.

Quantitative VS. Qualitative

Quantitative data is easy to understand. It’s numbers we track and measure. It’s easier to analyze. You can rank, compare and extrapolate quantitative data. With quantitative data, you can aggregate it, extrapolate it, and put it into a spreadsheet. But it’s seldom enough to give you a clear picture of your customers and your business. You can’t walk up to people, ask them what problems your product is solving for them, and get a quantitative answer. For that, you need qualitative input.

Qualitative data is messy, subjective and imprecise. It’s stuff of interviews and debates. It’s hard to quantify. You’re looking for the answer ‘why.’

Both give you a different perspective to your customers. When you marry your qualitative results to qualitative feedback, you have a clearer picture of your customers’ behaviors. When “what your customers tell you” and “what your customers do” agree with each other, then you’ve got a stronger signal as to what your customers want or need.

Known Unknowns VS.

Listen To Your Customers

Each product management tool has its strengths and weaknesses. Product teams should not depend on one tool too much. If you only use Google Analytics, MixPanel or Kissmetrics to guide your product decisions, that’s a great start, but it’s not enough.

For more product development tips, Subscribe to the ListenLoop email list or follow us on Twitter (@ListenLoop).