I’ve been watching a discussion on a LinkedIn Agile group about whether or not a team should assign points (or estimates generally) to a bug for inclusion in a sprint. I’m going to write about how our teams handled the question and think about it from an ‘agile values’ point of view.

The Question

I’m assuming a Scrum process here, so we’ll be talking about sprints, stories and bugs:

• A sprint is a fixed-time iteration of design, development and release.
• A story is an “increment of business value.” In other words, once this story is done, someone will be able to do something useful. It has a “level of effort” estimate, usually “points.”
• The velocity is the number of points per sprint, usually a weighted average over the last few sprints. It’s used for forward estimates of work that can be completed.
• A bug is a defect or unintended behaviour from the system. More generally, a ‘bug’ exists when the systems behaviour does not meet a customer’s expectations.

It should be relatively uncontroversial that if a bug is found in a story during its sprint then there’s no question what happens: the story card is simply “sent back” to the developer to be fixed[1]. The question some teams seem to struggle with is what happens when a defect is found after the story has been accepted as complete, and possibly even pushed to production. Here are some options people talk about when discussing how to handle the situation in an “agile” way[2]:

1. Take the original card, remove its “complete” status, and move it back into the backlog. Subtract its points from the velocity for that past sprint (this should lower the team’s average velocity, which is generally intended). A team may want to re-estimate the card, particularly if the bug appears particularly difficult or suggests that a deeper architectural issue or understanding of requirements is at fault. However this is generally avoided in order to maintain a consistent velocity.
2. Leave the original card, create a new “Bug” card, and add it to the backlog. If the card is urgent, it’s often added to the current sprint immediately. This approach can have two variants which are the subject of the LinkedIn discussion:
   1. The bug card is given an estimate (usually in points). If the bug goes into the current sprint, then the team is now either over-committed and likely to fall short; or the Product Owner is being asked which of their stories is the lowest priority and can it move to the next sprint? Either way this usually means that the team’s velocity for that sprint or the next is not affected, because the same number of points are delivered.
   2.  The bug card does not have an estimate or points. Teams usually pull it into the current sprint anyway, or agree with the Product Owner if it should wait for the next sprint. In any case, the team’s velocity is likely to fall, because the bug effectively counts for “zero points”, no matter how much work it takes to fix.

Our Answer

Our teams had the same discussions and we came down on the side of not assigning points to bugs. We saw points as a useful measure of how much functionality could be delivered each sprint. A bug was generally a fix to something already delivered. Fixing a bug may deliver “value for the customer,” but really what we were doing was “making good” on an earlier promise to deliver value. If there were a lot of bugs, velocity would drop. That was intended — it surfaced an issue in the development process. The root cause could be a poor choice of software library, too much technical debt, developer capability, or an inadequate QA process. By surfacing the problem, we were on the hook for addressing those issues, and could explain to management what was going on. Looking back though, I can see that it many ways, it almost doesn’t matter. What made the difference were the values that underpinned the discussion in the first place:

• Our highest priority was to satisfy the customer through early and continuous delivery of valuable software. Working software was our primary measure of progress.
• We wanted sustainable development, so that the entire business team could sustain a constant pace indefinitely.

We chose a process that we felt reflected those values, but you have to consider your own situation and adapt to it. (I might also note that “individuals and interactions” are supposed to be favoured over processes…)

More Light, Less Heat

These discussions sometimes seem to get a bit heated. I’m not sure if that reflects passion or pride, but either way my advice is not to put too much emphasis on the pronouncements of experts. They’re worth listening to, but only you know what’s going on in your organisation and your team and therefore what ought to work best. In one company I worked at, the accountants became aware of “points” and started including them in their reports. They would calculate the “cost per point” and developed an obsessive concern about reducing the cost per point and increasing the points per sprint (velocity). In that kind of environment you’re sorely tempted to assign points to everything. Worse, the team starts to suffer a kind of “points inflation” where the estimates start going up, velocity starts going “up” and “cost per point” starts falling — despite the fact that things are getting worse, and not better.

This makes me want to try an experiment one day: what if developers didn’t estimate level of effort on cards at all? Instead, product owners estimated how much additional revenue stories will earn the business per annum. The estimates would be in order of magnitude, for example:

• Fixing a CSS bug on a particular browser won’t do much, so it’s worth $10 pa.
• Fixing a CSS bug that affects all browsers might do a bit more. Maybe it makes the site look more professional. Anyway, that might be worth $100 pa.
• A very small amount of fraud is coming from a particular location. Block it to save $1,000 pa.
• A small percentage of customers want to save their shopping carts as wish lists. We might estimate this would add $10,000 pa.
• Expanding wish lists to wedding registries could bring in $100,000 pa.
• An entire new line of business might bring in $1,000,000 pa.

An alternative would be to use ‘percentage increase’ in revenue instead.

Either way, a team’s velocity would be calculated based on business value, rather than level effort. The assumption here is that business value is roughly correlated to level of effort over the medium to long term. My hypothesis is that teams might change the way they do things:

• Product Owners and developers are much more focused on delivering business value.
• Large stories would have to be broken down into smaller increments of business value. A story worth “$1,000,000 pa” won’t fit and so we focus on doing something that will meet some customer needs and bring in some revenue early.
• It would become much easier to practice simplicity: the art of maximising work not done.

No doubt there’d be unintended consequences too. Still, would like to try it. 🙂 Continue reading →