Velocity is not a Business Metric
Nowadays, every Agile™ organization is intently focused on increasing velocity. It's obvious: higher velocity means more productivity and efficiency, right? Because of this, managers and team leaders and product owners the world over are tracking team velocity. With it, they demand estimates, make projections, and kick off marketing campaigns. They are often led astray by velocity.
Because velocity is not a business metric.
Velocity is an imprecise metric in the best cases. We calculate it from an estimation metric over a defined period of time. Story points are the most commonly used metric. Story points are a team's best guess at how large a story will be relative to other stories. A story is assigned a number of points at some point before development starts on it. The points are credited to the team's velocity when the story is considered done. Ideally, velocity is fairly stable over time. Using velocity (say, 12 points/week), we can extrapolate an entire project (about 132 points) to determine the project's duration (132 points / 12 points/week = 11 weeks).
From that description alone, we see challenges using story points as a business metric. First, story points are relative and fuzzy. Developers are guessing how big a story might be before doing any work on it. It might be much smaller and simpler than expected. It might be much larger and more complicated. Even though these variations should smooth out in the long term, they introduce uncertainty in the short and medium terms. Teams do not stay stagnant. A maturing or changing team will change the baseline they use to guess story points. The value of a point will change over time. Projects also do not stay stagnant. A story point estimation of a story will not be consistent throughout the project.
Second, story points are team-specific. Different teams will make different story-point estimations for what are otherwise similar stories. Since the metric itself is not comparable across teams, velocity also is not comparable. Directly ranking teams by their raw velocities misleads the business on the true output of each of the teams. Teams competing on velocity will simply increase the points-per-story. This will increase their velocity while not actually changing their productivity.
Some business try to overcome these shortcomings by standardizing. One point is half a day, two points are a whole day, etc. Therefore, every member of every team completes about 10 points every week. We've solved the problem with variation. Or have we? Different teams produce at different rates for many reasons. Dictating the points does not change that fact. Teams will make up whatever numbers they need to so the business sees what they want to. Velocity looks approximately constant, but actual production varies wildly. The ability of the team to detect and diagnose their own problems from their velocity is lost. The business has actually made things worse.
Velocity can be useful to a team to measure themselves over time. They can track changes in productivity, determine what is causing those changes, and make adjustments accordingly. The business outside of the team cannot see this with velocity. Their reactions would be ill-informed and therefore ill-advised. The business instead needs to focus on meaningful business metrics. The primary metric should be the delivery of business value. Business value is the only thing that matters to the business. The biggest shortcoming of story points, and therefore velocity, is that they measure estimated effort. But effort does not always correlate to value. A smaller task may have a high value while a larger one a small value. The interest of the business is encouraging the team to deliver that value. So they need to focus on that, not velocity.
Because velocity is not a business metric.
Velocity is an imprecise metric in the best cases. We calculate it from an estimation metric over a defined period of time. Story points are the most commonly used metric. Story points are a team's best guess at how large a story will be relative to other stories. A story is assigned a number of points at some point before development starts on it. The points are credited to the team's velocity when the story is considered done. Ideally, velocity is fairly stable over time. Using velocity (say, 12 points/week), we can extrapolate an entire project (about 132 points) to determine the project's duration (132 points / 12 points/week = 11 weeks).
From that description alone, we see challenges using story points as a business metric. First, story points are relative and fuzzy. Developers are guessing how big a story might be before doing any work on it. It might be much smaller and simpler than expected. It might be much larger and more complicated. Even though these variations should smooth out in the long term, they introduce uncertainty in the short and medium terms. Teams do not stay stagnant. A maturing or changing team will change the baseline they use to guess story points. The value of a point will change over time. Projects also do not stay stagnant. A story point estimation of a story will not be consistent throughout the project.
Second, story points are team-specific. Different teams will make different story-point estimations for what are otherwise similar stories. Since the metric itself is not comparable across teams, velocity also is not comparable. Directly ranking teams by their raw velocities misleads the business on the true output of each of the teams. Teams competing on velocity will simply increase the points-per-story. This will increase their velocity while not actually changing their productivity.
Some business try to overcome these shortcomings by standardizing. One point is half a day, two points are a whole day, etc. Therefore, every member of every team completes about 10 points every week. We've solved the problem with variation. Or have we? Different teams produce at different rates for many reasons. Dictating the points does not change that fact. Teams will make up whatever numbers they need to so the business sees what they want to. Velocity looks approximately constant, but actual production varies wildly. The ability of the team to detect and diagnose their own problems from their velocity is lost. The business has actually made things worse.
Velocity can be useful to a team to measure themselves over time. They can track changes in productivity, determine what is causing those changes, and make adjustments accordingly. The business outside of the team cannot see this with velocity. Their reactions would be ill-informed and therefore ill-advised. The business instead needs to focus on meaningful business metrics. The primary metric should be the delivery of business value. Business value is the only thing that matters to the business. The biggest shortcoming of story points, and therefore velocity, is that they measure estimated effort. But effort does not always correlate to value. A smaller task may have a high value while a larger one a small value. The interest of the business is encouraging the team to deliver that value. So they need to focus on that, not velocity.
Comments
Post a Comment