This post covers how to think about what metrics matter the most, and my recommendations on the most important success metrics.
Context is key
Evaluating the success of a product education program is a topic that I see come up often from colleagues in the SaaS industry. And for good reason—it’s difficult to accurately capture the impact that it can have. That’s why it’s critical to focus in on a few key metrics (especially early on in your product’s lifecycle) as your North star.
As with most things in life, context matters. There’s no singular metric that can define the success of product education. Your North Star will vary depending on your product direction, company goals, how and where you educate your customers, and organizational structure.
Where that team is positioned will heavily influence how their impact is measured. For example, a product education team could easily sit within the product, success, support, or even engineering organizations in your company. Or for early-stage companies, it may just be the responsibility of one person who wears many other hats.
Lets think about this from a different perspective—where and how your content is distributed. Your education program could span across a variety of experiences, including Help Centers, Learning Management Systems, internal wikis, in-app experiences like onboarding cards or embedded content, social media, live events like webinars or training sessions, and so much more.
Prioritize impact over activity
Given the complex nature of measuring success, here’s the true North Star that you should prioritize; prioritize impact over activity. The success of a successful product education program lies not in how much work was done (activity), but the difference it makes (impact).
It’s less important, for example, that a content team made over 100 updates to your knowledge base last month. What’s more important is whether or not those updates are (1) relevant to customers and (2) effectively provide knowledge and resolution to their questions.
Now, let's break down the difference between activity and impact with a few real-world examples.
That doesn’t mean that activity isn’t important—it just tells a different story. Using another example, knowing how much your customers are using your help center search can inform how you structure your content. If 80% of unique visitors use search, you’ll know that you need to:
Orient your landing page to highlight the search experience
Heavily prioritize content that addresses common searches
Measure search click-through rates, searches with no results, top trending searches, and more.
On the other hand, if a minority of customers utilize search, you might:
Focus more on making navigation more intuitive
Visually bucket content into easy-to-identify categories
Split up long-form content into shorter, more digestible articles
These examples highlight the fact that what you measure needs to be informed by what’s most important to your organization.
Categorizing metrics
One way to help determine what matters is to bucket metrics into a few high-level categories to help explore what’s most important within them. Here are the categories that I’ve found are most helpful:
Creation: how much content is created
Delivery: when, where, and how content is distributed
Engagement: how customers are interacting with your content
Resolution: how effective your content is at meeting your customer’s needs
Satisfaction: how happy customers are with their content experience
Starting from these categories, you can narrow down what matters most to you within each of them. For example, looking at just engagement, you might discover that these metrics are most important for you.
The top three metrics that matter
No more beating around the bush—here are the metrics that matter. Based on conversations with colleagues, I’ve found that most people approach this from an organizational perspective, so I’ve grouped the top-three metrics based on where an education team might sit within an organization.
Customer Support
Customer Success
Marketing
Product
Engineering
What metrics matter to you?
I’d love to hear which metrics are the highest priority for you and your organization, especially if it’s something not found in this post. Feel free to drop a comment below or email me directly.