The Most Important Metric

Meeting Jeff Sutherland

Yesterday, I was lucky enough to meet Jeff Sutherland. I had contacted him in the hopes that he could point me in the right direction with my PhD. It turned out that he was in Amsterdam – not far from where I live – and he was willing to meet with me. I met with him in a cafe and after ordering lunch, we started talking.

Talking about my love for efficiency

I started by introducing myself and told him about my life as a developer and my passion for efficiency. We talked about the first great team that I was ever a part of and many other teams that I had been a part of. We visited the idea that some teams are very efficient and some are very inefficient. Jeff pointed out that there exists data that suggests that the difference in efficiency between teams can be as big as a factor of over 2000.

As we were talking about my experiences with different teams, we came to the subject of the last team I worked in before starting my PhD. In this team, we valued getting metrics about our own performance. Similar to what is described in the Kanban Guide for Scrum Teams. We also measured a metric that we didn’t have a name for. It was the ratio of the time we spent working on a story to the total time it took to get the story in production. Our ratio was horrible (0,7%). This was astonishing to us because we thought of ourselves as one of the most productive teams in the department.

Introducing Process Efficiency

Jeff told me that the metric I had been talking about is called process efficiency. He said that he thinks it is the most important metric for any team to measure if they want to speed up their development cycle. As an example, he told me about a company where he had been a consultant. He had given the advice to start measuring this metric and display it publicly. Very soon thereafter, the teams started moving software into production much faster.

This was very interesting to me. Because this metric is what I want to focus my PhD research on. I believe that optimizing this metric will in fact not only accelerate teams. It will also create an environment where teams can benchmark their own performance. They can get a clear idea of how much more efficient they could be. It will give management a way to detect whether or not teams are improving in a very concrete way. It will also incentivize managers to let developers do whatever they think is best to optimize this. And quantitatively show the results. Most importantly, I think that this metric cannot be gamed in any kind of meaningful way. If you improve this metric, you will improve your delivery.

What now?

I intend to gather data on teams that use this metric. My hypothesis is that focusing on this metric will accelerate teams like it did on my last team. I want to see if real life data can support this assumption because a well-founded claim makes a convincing argument for teams to start improving their efficiency in this manner. If a thing is truly good, the data will show it.

We both thought that “80% of agile coaches don’t use data to back up their claims. The other 20% is right”.

By doing the work, collecting and analyzing the data. I hope to show that process efficiency is the most important metric for any team that wants to achieve high productivity.

 

As a final word, I would like to thank Jeff for taking time to meet up with me and discuss the topic. Thanks.

Jeff and Frank
Jeff and Frank

10 Replies to “The Most Important Metric”

  1. Hi thank you for your research word. It‘s very important for the Scrum professionals. In fact I agree that most Scrum Master don‘t focus on process metrics. So fare I understand and expirienced in several software project to see Scrum as a production system as well as a CAS is the best way to figur out suitable process metrics like „acceleration“, „through put“ but also a metric for the amount of happiness.

    1. Hi Ronald. I am very happy to be doing this research work 🙂 I feel blessed that I have the opportunity to help improve the lives of IT professionals. As my work on the research goes on, I will keep you updated on this blog.
      Cheers

    1. Hi Frank. That is indeed a very good point.
      For the simple answer, I would point you to a paper by Carsten Ruseng Jakobsen and Jeff Sutherland, called “Scrum and CMMI – Going from Good to Great: Are you ready-ready to be done-done?”. They reference measuring process efficiency in a CMMI Level 5 organisation at the end of the paper.

      For a more thorough answer, we are going to write a paper about how to measure this metric. It will be the first one, but we want to do a good job, so it might take a little while. I will keep you posted here on the progress that we make.

      1. Hi Frank and Frank 🙂

        I have started the use of process efficiency for one of our squads (our leadership teams’ sprint) within KPN iTV in the Netehrlands.

        I use the following…

        The formula s as following
        Process efficiency = (Storypoint_story/Storypoints_Sprint) / (x/days_in_sprint) * 100

        X = StartTime_DONE – StartTime_InProgress

        Process efficiency = (Sp_story/sp_total) / ((t_DONE – t_InProgress)/days_in_sprint) * 100

        Frank Verbruggen, would you have time to share ideas about this topic and to see how it influences teams and the team dynamic?

  2. The metric that you refer to is called flow efficiency. The Kanban community has been using that metric extensively in the last ten years. Any Lean Kanban University trainer or coach can point you in the right direction. You may also want to check Niklas Modig’s TED talk and book “This is Lean”

    1. Patrick, that’s great!

      I’m very interested in showing that flow efficiency should be the main driving force behind optimizing the way IT departments drive efficiency.

      Especially in agile organisations, I see a lot of bad things happening with regards to flow efficiency. I will focus on trying to raise awareness in the agile community about flow efficiency, optimizing it and show how to integrate it into scrum so that we may achieve better results.

      I also want to quantify with data what the results are for these companies from an IT perspective. More on all of my endeavours will be posted here.

      Thanks for pointing me in the right direction!

    2. Well put Patrick. I find it interesting that some folks in this day and age are still “rediscovering” these subjects. We need to stabilize our terms.

      It’s Flow Efficiency. And it matters that we get our language stable because otherwise people WILL get confused and more harm than good could result.

      That being said, I am glad that folks are “discovering”, and hungry to know more.

  3. Did not understand the processs efficiency formula…can someone explain in simple terms?

Comments are closed.