Metrics provide, without any doubt, some of the most valuable information to businesses and their products. The question is - which ones should we use?
We live in a world where measurements and information are abundant, be they heartbeats per minute, number of application downloads, or social media followers. What’s crucial is to learn what information is relevant and how to use it.
The goal of this blog post is to take us through two Product Frameworks that will help us select the ideal metrics for the right product/company maturity. They break down metrics into categories, which will, in turn, help us define a metrics strategy and consequently start measuring a product’s performance.
This data is then used to experiment further, iterate and improve our products to deliver more value to users and the business.
AARRR Framework 🏴☠️
This user-lifecycle-centered metrics framework, created by Dave McClure, focuses on assisting teams in defining metrics that will help them get insights from the users’ journeys to increase revenue.
It’s more adequate for products and companies in their early stages, as it focuses more on the business side of things. During their early days, companies and products need to create a solid user base and generate revenue as soon as possible. This framework helps guide them in that sense, helping them be more effective and efficient.
This framework, which can be seen as a funnel, divides metrics into the following categories:
Acquisition — How do users find the product?
These metrics will help you understand how your users are finding the product. They will help you know what Marketing strategies, such as SEM and Refer-a-friend campaigns, are paying off. Within these Marketing strategies (Social Media, Blog, SEM, SEO, Paid, etc.), you may find metrics related to:
- Devices being used (Desktop, Tablet, and Mobile)
- Sources of traffic (Direct, Social Media, etc.)
- Landing Pages (Conversion Rate, Bounce Rate, Page Views, etc.)
Activation — Are visitors becoming users?
These metrics will aid you in understanding conversion results. With these, you’ll be able to identify if you are successfully turning visitors into users. Typically, this happens when users have reached a level of engagement with the product that makes them feel it’s worth their investment — the “Aha” moment. This could be a user purchasing a license after a product trial. Examples of metrics that will help you with this are:
- Click-through Rate (CTR)
- Number of Signups
- Exit Pages
- Number of purchases from new users (e.g., eCommerce)
- Number of users with more than 10 posts (e.g., Social Media)
- Page Views per Session
Retention — Do users come back?
These metrics aim to bring insights into how well your product keeps existing users. Keeping existing customers is cheaper than acquiring or selling to new ones, so this should be a priority. In this realm, you’ll find metrics such as:
- Number of Returning Users vs. New Users
- Time to Churn
- Average Time on the App over X days
Referral — Are users referring the product to others?
These metrics give visibility to how/if users recommend your product to others. It’s widely believed that word-of-mouth is the most effective form of Marketing and, thus, an extremely important means to increase adoption. Examples of these metrics include:
- Net Promoter Score (NPS)
- Percentage of Customers Referring Friends
- Social Media (Number of social mentions)
Revenue — How can you increase revenue?
These metrics’ purpose is to help learn how a product’s doing revenue-wise and consequently what can be done to increase that value. Examples of these metrics are:
- Customer Lifetime Value (CLV)
- Customer Acquisition Cost (CAC)
HEART Framework ❤️
This user-centered metrics framework, created by the Google Research Team, aims to help define metrics to be used to track progress toward specific goals.
It’s better coupled with products that have had some market validation and within more mature companies. This is because it’s quite granular and can be used for specific features and not necessarily for the entire product. The HEART Framework divides metrics into categories, detailing:
Happiness — How satisfied?
Users’ state of mind or disposition can be interpreted via their thinking, feelings, or behavior. They are usually collected via surveys and relate to aspects such as satisfaction, likelihood to recommend (NPS), visual appeal, and ease of use.
Engagement — How often?
What is the users’ level of involvement? This translates into the frequency, intensity, or depth of interaction over a period of time. These are usually collected by Analytics software and should be calculated on an average per-user basis, as higher results can mean more users and not more engagement. Examples are:
- The average number of likes per user per day
- Average page views per user per week
Adoption — How quickly?
A product or its feature’s performance in acquiring new users. The number of accounts created per week is an example of an adoption metric and often is measured via Analytics software.
Retention — How long?
How many users from a given time cohort are still users in a later time period? An example is a percentage of daily/weekly/monthly active users in a period (e.g., January) that are still active users in another period (e.g., February). Engagement metrics help predict Retention results, with higher Engagement results usually meaning higher retention.
Task Success — How easy?
How efficient and effective is a given task? These metrics give us quick indicators of possible improvements, are used primarily in specific processes (e.g., signups), and are particularly important for A/B tests. These metrics include:
- Time to Completion
- Percent of Tasks Completed
- Error Rate
To put this framework into practice, teams should define a Goal for each category, identify Signals that indicate success for each goal, and only then select Metrics to track results. An example of this could be a Food Delivery app that’d launch a feature that facilitates users re-ordering food they’ve ordered.
What’s next? 🏃
Once you’ve used one of the above frameworks (or a different one) to define metrics for your product, the next step is to use the insights these wield to implement improvements and make changes.
At this point, one needs to deep-dive into the source of the result of each metric and possibly resort to initiatives such as User Interviews to gather more information, or A/B testing to validate alternatives.
It’s vital to keep measuring each metric consistently, as having a large data set will help draw more accurate conclusions over time.
Conclusion 💡
It’s crucial for any Product Owner or Product Manager to have well-defined metrics for their products and features. Having visibility of how a product performs within the different verticals (e.g., Retention) gives a broad view of which areas need more attention and, thus, helps deliver a better experience to our customers and more value to the business.
Now let’s see how well this post does in terms of Engagement, Referrals, and Retention… 📈
Bibliography 📚
- Startup Metrics for Pirates: AARRR!!! [David McClure, 2007]
- Measuring the User Experience on a Large Scale: User-Centered Metrics for Web Applications [Kerry Rodden, Hilary Hutchinson, and Xin Fu, 2010]
