Experience Metrics Framework
What is it?
A set of metrics used to measure user experience in a consistent way that enables benchmarking against industry, peers, other systems, and over time.
The metrics are built into a modular survey template, and include a recommendation for “minimum metrics” for benchmarking. Additional task-level metrics are defined for collection during usability testing.
Why does it matter?
Consistently measuring experience allows us to benchmark across time, other systems, and industry, enabling us to measure improvements made.
What is included?
The experience metrics survey template includes:
- System Usability Scale (SUS) – industry standard tool to measure perceived usability
- Overall satisfaction
- Aesthetics – measures attractiveness and simplicity
- Net Promoter Score (NPS) – widely used in industry to measure loyalty & delight
- Features-level satisfaction
- Task metrics (frequency, importance, satisfaction)
- Open ended responses (what works, what could be improved)
Usability testing metrics include:
- Task completion rate
- Error rate
- Time on task
- Task ease of use rating
Minimum metrics for benchmarking
System Usability Scale (SUS)
The industry standard SUS consists of ten statements to which participants rate their level of agreement. SUS is intended to measure perceived usability rather than diagnose specific usability problems. It can be used as part of a usability test or in a stand-alone survey for benchmarking. Sample size and reliability are unrelated, so SUS can be used on very small sample sizes and still generate reliable results.
The Overall Satisfaction portion of the survey asks users to rate their overall satisfaction with the product on a 5 point scale. The survey includes a follow-up open-ended question about why the score was given.
The Aesthetics portion of the survey asks users to rate the attractiveness and simplicity of the product on a 5 point scale.
Net Promoter Score
Net Promoter Score (NPS) is used to measure customer loyalty and is most effective when a user would consider the system optional. NPS has been widely adopted with more than two thirds of Fortune 1000 companies using the metric. There is a high correlation between NPS and the usability of a product.
Features Level Satisfaction
The Feature Level Satisfaction portion of the survey is optional and asks users to rate their satisfaction with specific features of the product on a 5 point scale. For each feature, there is a follow-up question about why the score was given.
We combine frequency, importance, and satisfaction for specific tasks into an “Impact Opportunity Score” to determine the best opportunities for improvement of specific user tasks within the product that will have the greatest impact on improving the experience.
Additional task-level metrics
The following metrics are collected during usability testing and are not part of the survey template.
Task completion rate
The percentage of users that are able to complete a certain task
The percentage of errors made by users
Time on task
The average time required to complete a certain task
Task-level ease of use
Users are asked to rate the ease or difficulty of each task
If you are interested in gathering experience metrics for your system, please get in touch and we'd love to help you.