4 Common Mistakes while defining Success Metrics [PART 1]

TechnoManagers is the complete guide for Product Management, Quant Interview preparation and Investment Strategies.

This is part 1 of the article.

For a product manager, Metrics are as crucial as ideation of any feature or product. Metrics tell you the perfect picture of the different phases of your product/features. Defining the proper metrics and tracking them is extremely important for any product manager.
Before success metrics, we focused on the ultimate goals like revenue by-product, but it is a lagging indicator; the product may fall early, and the revenue is hit later, so relying on the end goal like revenue is not advisable.

We have a different framework for metrics like AARRR (Acquisition, Activation, Retention, Referral and Revenue), HEART (Happiness, Engagement, Adoption, Retention and Task Success) and North Star metrics.

This article aims to make you familiar with all these frameworks and then internalize some of the examples for a better understanding of the metrics.

AARRR Framework

Acquisition — How users are you finding your product?

Activation — How quickly do the users interact with your product?

Retention — How long users are using your product?

Referral — How often your users are recommending it to others?

Revenue — How much your users are paying for the product?

HEART Framework

Happiness — How happy are your user with the product?

Engagement — How often your users are using the product?

Adoption — How quickly do the users interact with your product?

Retention — How long users are using your product?

Task Success — How often do they complete the task which they are meant to do?

North Star Metric: This is the most critical metric directly related to the core value your product is providing.

Some common mistakes while defining metrics are:

1. Defining vague metrics: Defining a metric like “daily active users” is vague because the interviewer doesn’t know the scale of operation. 200K daily active users can be a significant number for a startup and can be disastrous for social media giants like Facebook. So the correct metric would be like “% change in daily active users.”
2. Defining incomplete metrics: Defining incomplete metrics like user acquisition rate (New user/ Total number of users) does not give sense about the period. The correct metric should be like - the rate of change of acquisition in a week.
3. Not aware of context: While clarifying the question, you have narrowed down the scope of the problem to particular geography, say India and demography say Young professionals. Including this in your metrics will signal your presence of mind. For example, “% change in daily active young professional on Facebook in India.”
4. Missing out on the core metrics: Always ask yourself what is the core value proposition of your product and give metrics for that accordingly.

Let’s see different metrics at different stages of the product journey.

Acquisition: In the acquisition phase, we track all those metrics which give information about acquiring new users and the source of acquiring. If our acquisition metrics are performing very poorly, there can be two significant reasons for this.

1. The product isn’t fit for the intended market.
2. There is a communication gap between the marketing team and the product team.

The following metrics can be primarily used in the acquisition phase

1. % increase in the Application Downloads
2. change in the ratio of new users/ total users over a defined time period
3. % of New users acquired through different channels over a specified period of time

Activation: In the activation phase, we need to have metrics that show us a clear picture of how quickly our product’s users are interacting with it. Following are the metrics we can use in the activation phase

1. % change in the average amount of time users take to do specific actions like signup, setting profile photo, setting address etc.
2. % change in the adoption rate (Users who did some action / Total number of users). Action can be anything like adding things to the cart, making his/her first purchase etc.

Engagement: In the engagement phase, we need to have all those metrics that tell us how often our product’s users interact with it. In general, engagement metrics for the core action or features are so important that we assign our north star metrics related to it.

1. Increase in the average number of sessions by users in a day.
2. Decrease in the average time between two different sessions by users in a day.

Task Success: In task success, we generally define those metrics that tell how easy users find it to use a particular feature.

1. Increase in the ratio of people who have completed action/ Total number of people who started that action.

Happiness :

1. The ratio of the users who recommended the product / Total number of Users. Recommendations can be through different social media channels or referrals.
2. Average rating of your application out of 5 like rating on Google play store.

Retention :

1. Users who joined subscription/ Total number of users
2. Churn rate
3. Bounce rate

Revenue :

1. Revenue per user

-TechnoManagers