Wednesday, July 1, 2009

Efficiency - A Must Have Agile Metric As Important As Velocity

When I first picked up Agile development, I absolutely got hooked on the simple yet powerful way it described how to track your team's progress on projects with REAL data. Before I implemented Agile, I can't remember how many times business stakeholders or execs have tried the Developer-Used-for-My-Benefit approach, only for it to bite the organization hard in the nuts later. I call it the D-U-M-B approach for short.

CEO: "How's the project coming along Pat?"
Pat the Developer: "Uhhh - going well. I can crank this out in no time - easy!"
CEO: "Excellent - I'm going to give you a huge bonus and tell sales and marketing to start pushing it."

Uh-oh... I think we all know the end of this all too well-known scenario... :(

Gathering actual hard data while your project is running is the only practical way to measure the completion of your project - and Agile processes are all about collecting and charting this hard data, making it visible to the entire organization so there is never any question where a project status is at!

One of the core hard data metrics within Agile practices is the team's velocity - the amount of work the team is able to complete in a given amount of time/iteration. Using the team's last velocity, the team can predict what their velocity will be for the next iteration. Charting the team's velocity for the project will allow you to make some smart data-based decisions as well by analyzing their velocity trends. For tips on providing your Agile team the best chance on achieving a high, stable velocity, read this later!

However, even with all the great info velocity charts provide, I found myself wanting something more than just the velocity chart. If resource days are limited in an iteration due to holidays, vacations, or sick days, the team's velocity can drop significantly. It makes complete sense of course - fewer working days = reduced work completed for that iteration. But, how could I prove that the dip in velocity was just due to the limited number of resource days? How could I prove that the team worked just as hard as they would have if they were fully staffed for the entire iteration?

Enter the efficiency metric. The efficiency metric allows you to determine how efficient the team was for each iteration, but in a way that it factors out the number of resource days that were available during that iteration. Thus, even if there was an iteration when the entire team took a few days off, where the Velocity Chart would show an obvious dip in the amount of work accomplished, you can rely on the efficiency chart to show if they were just as efficient working that iteration from the previous ones. Below is an example Team Velocity and Efficiency chart based on data collected for the same iterations.

You can see that even though the team's velocity dropped signficantly during iteration 14 because half of the team was out due to the flu for a few days, the efficiency chart shows that the team remained consistent in its effort to get work completed - which tells you that during that iteration, a dip in velocity was entirely due to the lack of resources that were available that iteration.

COOL BEANS GRANT - I GOT IT!! So how do you calculate a team's efficiency?

E = V/R

E=Efficiency for the iteration
V=Velocity for the iteration
R=Total number of person days worked on the team for the Iteration

Hopefully, you're already tracking velocity (V) for completed iterations. You'll also need to start tracking total actual person days (R) worked for your completed iterations as well. For example, if you have a team of 5 and your iteration is 10 days, then the maximum possible R value for your team would be 5 x 10 = 50 days. The actual R value would thus be the maximum possible R value for your team minus the number of absent days per team member during the iteration (e.g. 50 max - 1 sick day = 49). Tip: Plug all of your numbers in a spreadsheet and setup formulas to do your calculations for you! Efficiency is auto-calculated for me once I've entered velocity and resource days for the completed iteration into my spreadsheet. After you've got your efficiency calculated, producing your Efficiency Chart is as simple as producing your Velocity Chart - just chart it over each iteration!

Below is an example spreadsheet showing calculation of Efficiency using Velocity and number of resource days.

Now that you have your efficiency chart at hand, viewing it in combination with your velocity chart unveils much more information about your iterations than the velocity chart would have provided alone. See a dip in your velocity chart but efficiency was stable? Resource days must have been limited that iteration. Like velocity, the trend for the efficiency chart for teams heading in the right direction should be very similar to that of the velocity chart: new teams will likely have a few beginning iterations where efficiency may be erratic when they start settling into the project and learning to work together. But, eventually, you should see a peak and plateau, representing that the team's current maximum efficiency has been achieved. Note that efficiency ratings are not the same between different teams - so don't get all twisted up if one team seems to have a smaller efficiency than another! To me, a successful efficiency chart is not about how high it reaches, but about if it has reached a peak and consistent plateau.

Having efficiency also allows you to obtain what I'm calling the benchmark velocity for your next iteration. If you're using velocity charts right now, I'm sure you've been told that the best way to determine how much work the team can do in the next iteration is the velocity from the last iteration. I beg to differ!! With the efficiency metric of the last iteration in hand, simply multiply it with the expected number of resource days for the next iteration, and you've got the benchmark velocity - the velocity your team would hit if they work just as efficiently as they did this past iteration.

v = E*r

v = benchmark velocity for next iteration
E = efficiency from just completed iteration
r = Total number of expected resource days available for upcoming Iteration

Below is a spreadsheet showing the simple formula to calculate the benchmark velocity.

Grant - why do you call v the "benchmark" velocity instead of the "expected" velocity? After all, isn't that what it really is? When the team needs to decide what their upcoming velocity commitment should be, there are more things to consider than just the pure numbers from the previous iteration. For example, maybe your office is scheduled to move to another building next week. Even though everyone is still working those days, you shouldn't expect them to be just as efficient as they were last iteration!! Thus, you and the team should expect to have a lower velocity for the upcoming iteration than what the "benchmark" velocity is calculated to be... hence the term "benchmark" instead of "expected." Saves you the trouble in also explaining to project stakeholders, customers, and execs on why the "expected" velocity wasn't reached as well if you had called it that!

Allowing your team to view the benchmark velocity and then talk through any outside factors that could impact the upcoming velocity for better or worse should allow the team to settle upon a velocity number that they can commit to. A fully self-organized, steady team should more often than not accept the calculated team benchmark velocity as their commitment. If your team is consistently committing to numbers that are drastically different from the calculated benchmark, it's a clear signal that they are still learning how to stabilize their development efforts for the project. Don't be surprised if the team decides to commit to a velocity that is higher than the benchmark as well!

The efficiency metric is now a must have for all of my Agile run projects - it's my hope that your team and organization will also benefit from using this metric as well!