How to Become a Certified Scrum Master
Book a course with a Certified Scrum trainer, for example, ScrumCenter’s. After completion of the course, you will be sent a Scrum Alliance link to
Organisations run their agile transformations with teams staffed with trainers and coaches. They often track the progress of their work through the numbers of teams trained, coached and up to speed (sprinting). This is not without value, but at best, these are leading metrics for something much more important – the goal of the transformation.
The “why” of agile in a particular context might be about time to market, generating more innovation, responding to market forces more quickly, being more resilient in a crisis or delighting customers more thoroughly, or a combination of these things.
So make sure you also measure the “needle” moving on these things.
Both leading and lagging metrics are helpful. The leading metrics are not the goal but can help us to move the needle for things that count, such as delighting customers. The lagging metrics measure the goal more directly. Leading metrics are faster to change; lagging metrics are typically slower to change.
So, as well as measuring the number of teams using Scrum, make sure you measure the success of these teams, particularly the success of the products they develop.
Agile teams often track metrics in five categories:
The ultimate goal of an agile team should be to create and sustain satisfied, or even better, delighted customers.
A commonly used metric for customer satisfaction is Net Promoter Score (NPS).
NPS is based on the question “How likely is it that you would recommend [brand] to a friend or colleague?” using a scale from 0 to 10, where 0 represents not at all likely and 10 represents extremely likely.
NPS divides the scores as follows:
NPS is calculated as follows:
NPS = % Promoters – % Detractors
This is just one example of a metric for customer satisfaction, others metrics such as number of active customers are also very common. The main thing is to think about how you measure product success. Customer satisfaction is likely to be a large part of this, so adopt a metric which is easy and quick to gather and interpret. You don’t want large delays between shipping a new version of your product and getting data on customer satisfaction. Make sure that your organisation doesn’t introduce delays between shipping new versions of your product and getting data – this is very common in large, bureaucratic organisations and must be improved if this is the case for you.
An unhappy team will not be able to deliver great products sustainably. A common metric is team happiness, captured regularly by asking each team member to rate their satisfaction as a member in the team on a scale of 1 to 5, where 1 means very dissatisfied and 5 means very satisfied. Note this is effectively a measure of morale, it’s not a generic measure of personal happiness.
Capturing team happiness can be easily built into sprint retrospectives. It should primarily be something for the team to respond to – if happiness is low, a skilled Scrum Master will facilitate helping the team to discover why and what to do about it.
If you do publish happiness, for example, to management, make sure the data is anonymous. Otherwise, the team members will be less able to reveal their concerns during the retrospective.
Tracking product quality can help an agile team to get an early warning signal of problems before they result in a decrease in customer satisfaction.
A commonly used and important metric is the number of defects escaping into production, which basically should not happen! Tracking defects discovered during development is less valuable; discovering bugs is a standard part of development and shouldn’t be discouraged by turning it into a KPI!
Calls to a support or help desk or the number of support tickets raised are other commonly used metrics.
Many agile teams work, sprint by sprint, towards a product goal. A good product goal usually has a business case associated with it – the expected benefit might be additional revenue, cost savings, avoiding additional costs, preventing the loss of existing revenue or a combination of one or more of these things (see cost of delay).
Product owners should actively manage the return on investment of their products. There are various ways of measuring ROI, such as internal rate of return (IRR) and payback period. Try to align your agile teams and products with whatever ROI metrics are in use in your organisation. Whichever approach you use, you will need to measure value (which can only be measured after shipping) and costs (which you can easily measure at the end of each sprint).
Make sure you avoid asking the team to record engineering time, by filling in timesheets or any other method. This is wasteful and results in lower quality by establishing something allied to a “build to budget” or “build to what we estimated” culture. Instead, simply calculate the costs of a sprint, something that should be easy to do by combining day rates and days people spent working on the team during the sprint.
For teams which work towards a product goal, it can be important to report on the progress towards that goal, on a sprint by sprint basis. This enables the Product Owner to answer questions such as “how much is left to do?”, “which features can we ship within the agreed budget?” or “how many sprints do we need to complete this set of features?”.
Measuring velocity can be useful here but there are also some problems with velocity which I will describe shortly.
The velocity of a sprint is the sum of the story points of the product backlog items completed during the sprint. So, it’s a measured value.
By getting the developers to estimate the product backlog items in story points and measuring velocity on a sprint by sprint basis, the product owner can answer these questions. Creating such estimates can be part of product backlog refinement.
The problem with velocity is that it is often misused to compare teams: teams with higher velocity being judged to be better than teams with lower velocity. This results in a conscious or unconscious inflation of team’s story point estimates, which destroys any utility that velocity would otherwise have – self defeating in the extreme!
If all concerned avoid this misuse and use techniques such affinity estimation or bucket estimation, velocity can be a very powerful tool for agile teams, and, particularly for product owners.
A perverse incentive has an unintended and undesirable result that is contrary to the intention. When designing a set of metrics for agile teams, make sure you avoid the pitfall of perverse incentives.
One commonly seen example is measuring lines of code in an attempt to quantify developer productivity – what you will get is most likely, more lines of bad quality code!
In this article we’ve discussed some of the most common metrics used by agile teams and to measure agile transformations. We’ve also shown some of the common pitfalls.
This table summarises a starting point for a set of metrics for agile teams. Do be careful with velocity, however!
| Metric | Frequency | Leading or lagging with respect to “product success” |
| Net promoter score | Track continuously, pay particular attention to changes when you ship a new version of your product | Lagging. Likely to be well aligned with product success |
| Team happiness | Once a sprint, as part of the sprint retrospective | Leading in the sense that it sets you up for success but doesn’t directly measure it |
| Number of defects escaping into production | Track continuously. Often reported on as part of a sprint review | Lagging in the sense that quality problems have already occured. LIkely to be well inversely aligned to product success |
| Payback period | Manage on a product goal basis, usually based around major releases. Remember that return only happens after you have shipped | Lagging |
| Velocity | Measure at the end of every sprint | Leading |
Here are some links to what other authors have to say about this topic:
For IT operations or devops teams it is also worth looking at the four key metrics proposed in the book Accelerate:
At ScrumCenter we provide support for organisations, teams and individuals through our agile transformation consulting, coaching and Certified Scrum training.
Our interactive and engaging Certified Scrum Product Owner and Certified ScrumMaster courses both cover metrics.
Book a course with a Certified Scrum trainer, for example, ScrumCenter’s. After completion of the course, you will be sent a Scrum Alliance link to
Scrum “by the book” is very simple and it’s a great place to start, but teams that want to get outstanding results understand the journey has only just begun. The three pillars model can help your team on this journey by allowing you to maximise the effectiveness of each part of Scrum.
The three pillars of Scrum are transparency, inspection and adaptation. They can be used to “inspect and adapt” your Scrum implementation and set you and your team on the path to really excellent Scrum.
Measuring Agile Transformations Organisations run their agile transformations with teams staffed with trainers and coaches. They often track the progress of their work through the
We’ve moved our Scaling Agile Lean Coffee to the meetup.com platform. You can join the group here:
https://www.meetup.com/scaling-agile-meetup/
The upcoming events will be published in the group. At the time of writing, the next event is 12th May at 18:00 BST, 19:00 CEST. There is always a lively discussion so why not bring your burning topic on agile transformation or scaling agile to the group? It’s free to join.
Our Services
Help & Legal
About
ScrumCenter has supported large, medium and small companies with agile transformations, training and coaching in Europe and the USA since 2009. Our training is based on many years of experience using agile methods and helping organisations, teams and individuals achieve sustainable success.