Tuesday, February 24, 2015

Measurement Frameworks: Focus on the Metrics That Matter

There is certainly no shortage of data available to marketers, but the problem is often in knowing how to structure the data to get the most out of it.
In this age of "big data," we’re not short of numbers are we? We generally have access to vast amounts of data from all sorts of different tracking and analysis systems. We’ve got digital data, and we also have user data such as that from voice of the customer surveys and remote user testing programs, and increasingly we’re combining that with transactional and non-digital data sources. So we generally don’t have a problem getting hold of data anymore. For many the problem is in knowing how to manage it and to structure it in a way that allows it to tell us a story.
This is why we need to develop measurement frameworks. Measurement frameworks are a way of structuring metrics and those all-important key performance indicators (KPIs) around the strategy, goals, and objectives of the business. The key thing about a measurement framework is that it’s coherent and helps the business to understand the relationship between the metrics as well as the metrics themselves.
Different frameworks are needed for different use cases and there are various styles available. Balanced scorecards are an example of very strategic measurement framework, looking at the performance of an organization as a whole. Functional areas many have their own measurement framework to provide structure to their own metrics. A good example I came across years ago was a framework developed by some people at Google to understand the user experience of some of their products. They called it HEART.
HEART stands for Happiness, Engagement, Adoption and Retention, and Task Success. They used this framework to understand the user experience across their various different products and the important point is that the metrics that make up the framework came from a variety of different data sources. Happiness, for example, is measured using attitudinal metrics gathered using survey data; adoption and retention may be measured using customer data; and task success by using remote user testing methodologies. The actual metrics used to measure the user experience vary from product to product depending of the nature of the product. For example, the way that Gmail is measured is different from the way that Google Maps is measured, but the HEART framework is the rigour and coherence to the individual metrics and usually describes what is being measured and to some extent why. It also always helps to have a good acronym!
To develop a framework you need to go back to the objectives and determine what it is you need to measure and why. In a straightforward e-commerce scenario, for example, a measurement framework may be based around the transactional environment looking at sales, order, average order value, number of customers, number of orders per customer, and so on. The framework would highlight the relationship between the different variables and how a change in one might or might not influence the other.
Another scenario might be in customer service. In a previous role I was asked by a client to help them think about how to measure the success of various customer service and self-service initiatives. In line with many companies, they were looking to increasingly address their customer service needs using various online channels such as websites, live chat, forums, and so on. One typical measurement perspective would be to focus on some of the core digital metrics, such as visits to the customer service section, number of downloads, number of chats, etc. These metrics were certainly useful in terms of understanding what’s actually happening in the digital channels, but were not necessarily indicative of whether the initiative overall was successful or not. To do that we needed to develop a wider framework that looked at the customer service section in the context of the overall customer journey.
The framework included measurements that looked at to what extent customers knew about some of the digital services, whether they would use them or not, and what they thought of the experience. The framework essentially told the story about the migration of customer service from offline channels to online channels and allowed the organization to understand where the focus areas needed to be. Did the site need to improve or what is good enough? Did they need to focus on educating customers about the digital channels and the benefits they offered? The development of a specific measurement framework was able help the organization understand the most effective way to hit the initiative’s goals and objectives.
With data everywhere it’s easy not to be able to see the wood for the trees. Developing multi-faceted measurement frameworks is the way to add structure to your data and to really focus on the metrics that matter.

No comments: