The story a sprint burndown chart told me

They say a picture can tell a thousand words. If you understand what a sprint burndown chart means, a thousand words might not be quite enough. In my case, it wasn’t. Yet, at the same time, a few short words can also enough to start asking questions. ‘Something’s not right’. Three short words I said in my head started us down a path of a series of questions.

I have to confess something upfront. I hate sprint burndown charts. Velocity, story points and deterministic thinking also fit into my ‘hate’ category. At best, they’re waste. At worst, from my experience, they’re misused, misunderstood and aren’t a useful tool. Simply put, they can’t do what they promise.

They can give a high-level and, therefore, a superficial view of what is going on. They can lull teams and management into thinking everything is going well. When things have gone wrong, it’s too late anyway at that point. How many teams use KPIs that say something like ‘deliver X story points each sprint’ to measure success?

At their heart, they’re an indication of progress against a super-estimation built on a series of smaller estimations. Our very own house of cards.

Sadly, they’re also an established part of the Agile landscape. Burndown charts are widely used and understood. That’s the case in my organisation at least.

For me, it was a starting point to tell a bigger story. That story can be a powerful one if you know how to read a chart. I used them widely in the early days when I joined a team as a product owner (PO). They’re simple and effective in making a point. They can help teams ask the right questions and take the first steps in adapting and evolving the way they work.

Understanding what a chart will tell you

Before I share some actual burndown charts, it’s helpful to make sure we’re on the same page. A burndown chart will show you how much work is remaining within a sprint. Some teams will use an alternative burnup chart, which shows work completed against time. That then is usually a different view of the same information.

The Scrum Guide defines a sprint as ‘the heartbeat of Scrum, where ideas are turned into value. They are fixed length events of one month or less to create consistency.’ As you read on, you’ll notice burndown charts get a brief mention as a possible tool. Burndown charts are then not essential. In other words, you can be doing Scrum and not be using burndown charts.

In a sprint planning meeting, the team will discuss the work of the sprint and agree on a sprint goal. The work agreed becomes the sprint backlog. As the sprint goes on, a burndown chart shows work done vs work outstanding. It’s a line chart showing the amount of work remaining (y-axis) in the sprint backlog against time, usually days (x-axis).

Different teams will use different ways to quantify the amount of work, though this is often done through story points. A story point is an estimation by the team on the complexity of delivering a user story. Sometimes, teams will use it more crudely to map a point against time. For example, 2 points are equal to 1 days’ work. Some teams might use a Fibonacci scale to shape this.

So every time work is done in a sprint, the overall quantity of work remaining is lower. The chart’s burndown line then burns down — hence the name. The aim is to complete the final piece of work at the end of the sprint, delivering the increment and having the line finish at zero.

Atlassian, the creators of Jira, have a good tutorial on this which I recommend if you want a deeper understanding.

If everything is going well, the team will have a chart that looks like an even staircase, going down to zero. In general, this will tell you that the team have good work patterns, with a smooth flow. Like a well-oiled machine, or an orchestra playing a majestic symphony, the team works together from one task to the next. When they find an impediment, they resolve it. If someone needs help, the scrum master is there to help — or even better, the team self-manage to fix it themselves. The work committed to in the sprint backlog is finished in the time forecasted.

That’s the dream, though in a complex world, that is rarely is the case. This is why an empirical approach is needed over any tool, like burndown charts. All we have at any moment is knowledge and understanding on what has already happened to help guide us on a path forward.

How our burndown charts showed what was wrong

The following are three real charts from our sprints. Taken from Jira, I’ll tell you the story behind each one. How we got to that situation, questions we asked in sprint retrospectives to inspect and adapt, to work out what we were doing wrong.

For context, we worked to a two week sprint cycle. In our multidisciplinary team of UX, content designers and developers, our estimation was by story points, loosely linked to the expected time to deliver a user story.

Chart 1: are we making progress?

You’ll notice that nothing was really delivered until four or five days before the end of the sprint. The first week flatlines and by the start of the second week, the line is even going up a bit. We managed to pull it back in the last few days, though still finished short of where we had hoped.

The reason for the lag in the first week was because we didn’t manage our flow well. It was taking too long for work to enter and leave the system. User stories were being broken up by discipline. So for example, if we needed a new piece of functionality on a webpage, we would break the one user story into four different tasks (or Jira tickets in our case), which different team members would work on in isolation. People weren’t coming together as a team to working together on an issue. Handoffs could then be messy, particularly if people didn’t have a deep understanding of the issue. The work at the end of the sprint was often pressured to pull everything together into an increment we could inspect. Sometimes rework was needed to correct issues which should have been picked up earlier.

In most cases, the work would be completed, though not releasable without some more effort. We would then have to carry the item over into the next sprint. This meant our sprint review meetings weren’t as meaningful as they should have been, as we would have to discuss what more was needed in the next sprint.

Retrospective questions:

• How are you visualising the workflow? Are your user stories too large or the tasks too granular? Is our process too complex or have too many steps?
• Are you using the Scrum events well? Are the team self-managing effectively to ensure a sprint goal is on track?
• Are there impediments or bottlenecks? How can the scrum master step in, or better still, help the team to self-manage?
• Could we bring in some Lean ways of working, namely Kanban? Would moving towards a probablistic view of the world be of more help? Would using flow metrics or setting work in progress limits help? The Kanban Guide for Scrum Teams can help with this.

Chart 2: better sprint planning leads to better sprints

This sprint took a while to get going properly. You’ll notice in the first days, the burndown chart is going in the wrong direction!

We actually ended the sprint well, delivering almost everything we said we would. It just wasn’t the work we planned for in the sprint planning. The team had to bring in new work to deliver the sprint goal and then worked really hard over the sprint to do it all. Credit to them; they’re a great team. It just meant things were quite stressful.

To help the team going forward, we started to have more frequent and smaller refinement sessions throughout the sprint. Our sprint planning changed to focus on one or two big things for the sprint ahead.

Retrospective questions:

• Are we using the Scrum events properly, particularly the sprint planning meeting?
• Is the PO working with the team closely enough to properly refine the product backlog, ordering the right items by value?
• Are we selecting the right items for the sprint backlog? How can we make better decisions in the future?
• Are we managing stakeholders well and setting the right expectations on when we will do the work? Does the scrum master need to coach stakeholders to understand how Scrum works?
• A two-part question: first, is Scrum the best framework for the team? Second, could we enhance our Scrum with Kanban, particularly if the organisation’s environment is changeable?

Chart 3: have we over promised again?

This was a common chart for us. Notice the long flat periods, with small items either delivered or added to the sprint backlog. There’s a sharp drop at the end too. In this instance, the team managed to deliver a few more significant items right at the last minute. Though at other times, we had to take items out of the sprint backlog. The problem in this instance is the absence of anything to inspect at the end of the sprint. It’s a true mark of what the Liberators call Zombie Scrum.

Retrospective questions:

• Are we over-committing each sprint? Should we narrow the goal and the focus of the sprint?
• How is our stakeholder management? Are we waiting too long for feedback?
• Do the team have everything they need ahead of working on an item? Is there enough understanding of how to meet the definition of done?
• Are we spending enough time on our estimation and planning? Is that causing an overcommitment?
• Are we using the daily scrum effectively? What steps could we take earlier in the sprint to ensure we progress and raise issues earlier?

How we moved forward

By asking all these questions, we could get under the skin of what was going on. These conversations gave us insights burndown charts could never provide. It was empiricism, not burndown charts, that helped us become a better team. The three pillars of empiricism are transparency, inspection and adaption. Our journey took us in a new direction, one where Kanban and flow helped us deliver better and more valuable outcomes for our product.

Burndown charts told me something was wrong. It was an empirical approach that helped us moved forward. We’ve since stopped burndown charts and use flow metrics instead.