“In everyday life, the Flaw of Averages ensures that plans based on average customer demand, average completion time, average interest rate, and other uncertainties are below projection, behind schedule, and beyond budget.” – Sam L. Savage, 2009 – The Flaw of Averages
“Variation is the hard reality, not a set of imperfect measures for a central tendency. Means and medians are the abstractions.” – Stephen Jay Gould, 1985 – “The Median Isn’t the Message”
“Essentially, all models are wrong, but some are useful” – George E. P. Box, “Empirical Model Building and Response Surfaces”, (1919 – 2013)
“Remember that a model is not the truth. It is a lie to help you get your point across.” – Sam L. Savage, “The Flaw of Averages”, 2009
“You are allowed to lie a little, but you must never mislead.” – Paul Halmos, mathematician, (1916 – 2006)
Last week I read an article about yet another tool showing the ability to produce CFDs (cumulative flow diagrams). Maybe you’re already a user of one of these tools that visualize your workflow and generate them automatically (or “automagically”) as part of the reports they offer. Or, perhaps like me, you still generate them mostly using MS-Excel. Either way, have you wondered just a little about how a CFD works?
As most do, this article displayed a line extending vertically between “stages” on the CFD (or workflow processes, as I often call them) and identified this distance as the WIP (work-in-progress) on a specific date for the stage (or stages) of interest. There was also a description of another distance, a line extending horizontally between stages of interest on the CFD and identifying it as the “average lead time” for the requests (work items) arriving on a specific date. That is, the average time for a request (work item) to “flow” through (arrive into and leave from) one or more stages of interest. Lastly, there was a description of a sloped line, as a mean delivery rate of requests (work items) flowing into or out of a stage (workflow process) depending on viewing the workflow upstream or downstream. Note: for more on the basics on reading a CFD, see this earlier post here.
A Difference Is Not An Average, Right?
It is easy to understand that WIP is simply a “difference” between two counts on the CFD and represents the number of requests (work items) at a point. Similarly, seeing the slope as a simple rise over the run calculation of a number of requests per unit of time (a rate) is not a complex concept to accept. But, what about the notion of the “average lead time” derived from the CFD? How is it a “difference” between two points in time read from the CFD (ex. calendar dates) can represent an “average” unit of time for a request (work item) to flow through a stage (workflow process)? A “difference” that represents N numbers summed up and divided by N. Yes, really! But how can this be?
“The instinctual shortcut…when we have ‘too much information’ is…picking out the parts we like and ignoring the remainder, making allies with those who have made the same choices and enemies of the rest.” – Nate Silver, American statistician, writer. “The Signal and the Noise: Why So Many Predictions Fail – but Some Don’t” (2012)
If you want to be competent at continually uncovering the unknown, strategic processes need to be based on learning not planning…in the fast-paced world of twenty-first-century business, the focus of strategic work is no longer about making decisions – it’s about making discoveries. It is about discovering what you don’t know that you don’t know.” – Rod Collins, author, speaker, consultant; formerly Chief Operating Executive at Blue Cross & Blue Shield Federal Employee Program. “Why Planning Is No Longer a Strategic Activity”, (2013)
Over the last several months I’ve gratefully appreciated opportunities to present at conferences and user groups in Denver, Chicago, Houston, and Boston, and will have one more opportunity to close the year presenting in Pittsburg come Dec. While discussing possible topics with organizers of these events, most felt a presentation discussing some myths and misconceptions about the Kanban Method would be of interest and benefit for their attendees. [Note: you can find additional context, and a bit about the initial motivation for talking more about these myths and misconceptions in my earlier post titled “The Kanban Method: Is It Just Scrum With Tweaks or Is There More?”]
The latest variation of my presentations on this topic was at the Agile New England user group in Boston, where they capture many, if not all of their monthly presentations, on video and make them available publicly. If you’re in the Boston area and not familiar with ANE yet, I’d encourage you to visit their website here.
To assist with viewing the video I’ve listed a mini-index (below) indicating a few logical partitions where one might stop and come back to if you’re unable to sit through the presentation in a single sitting. [Note: this particular presentation allowed for a number of questions to be taken throughout, and not just at the end.] I hope you find it interesting. As per a quote I heard once somewhere that has really stuck with me, “the greatest learning occurs at the interface of disagreement” (adding, provided there is some constructive dialogue). So, if you feel I’ve misstated something, could articulate something a bit more effectively, or you disagree with something, please let me know.
Logical partition points after intro material:
07:23 – A taste of LKNA2013 from Chicago via Boston
07:56 – Getting on the same page with the term “kanban”
10:53 – Why is this important?
13:13 – If you take away “one” thing only
14:40 – First of the “myth and misconceptions”
17:49 – Second of the “myth and misconceptions” (including a number of questions from attendees)
42:57 – Notes on the term “iterate”
43:50 – Notes on decoupling activities
45:36 – Third of the “myth and misconceptions”
57:27 – Closing Q&A
I want to thank again here a few of the people from ANE who work hard on a volunteer basis to provide this first class user group in the Boston area. Thanks to ANE Member Shyam Kumar for his persistence, as we started talking about this opportunity some time ago and patiently working through schedules. Also thanks to ANE’s President, David Grabel, ANE’s Vice-President, Tom Woundy, and ANE’s Program Coordinator, Ron Morsicato, for their efforts that evening and prior, contributing to a first-class user group operation. Lastly, thanks to ANE Member Ron Verge who videotaped, edited and put together the video.
“Errors using inadequate data are much less than those using no data at all.” – Charles Babbage, English mathematician, philosopher, inventor, mechanical engineer, invented the first mechanical computer (1791 – 1871)
“I always avoid prophesying before hand, because it is a much better policy to prophesy after the event has already taken place.” – Winston Churchill, British orator, author, and Prime Minister during WWII (1874-1965)
“Policies are many, Principles are few, Policies will change, Principles never do.” – John C. Maxwell, evangelical Christian pastor, speaker, and author of 60+ books primarily about leadership (1947 – )
“We’re entering a new world in which data may be more important than software.” – Tim O’Reilly, founder of O’Reilly Media, supporter of the free software and open source movements (1954 – )
This question below came up recently on our Agile Denver Kanban SIG (special interest group) LinkedIn discussion list.
“I am working for an organization that has ‘Lab Week’ six times a year. Lab Week encourages innovation by setting aside time for engineers to work on whatever they want. My current conundrum is, what do I do with the work items on my team’s Kanban board, specifically those in To Do or Doing, during Lab Week?
It feels wrong to put them back in the backlog, but keeping them in To Do or Doing will affect lead/cycle time. While this is reality, it is only reality six times a year. In the interest of predictability I think I would want to know what lead/cycle time is without the impact of Lab Week and then factor in Lab Week when we hit a Lab Week.”
I had two immediate responses to this interesting question. First, software development teams in a number of IT/IS organizations are familiar with the basis for this question, and I’m guessing it comes up in a number of non-IT/IS or non-software development contexts as well.
Second, there is already an upfront discomfort with the idea of moving work items back to the “backlog” that have been “pulled” into the ToDo (or Ready, On Deck, etc.) workflow process states. I’m wondering what signal is this discomfort providing to us? I’m guessing the discomfort is even greater with the idea of moving work items back to the backlog that has made it to some state of Doing in the workflow process. In essence, this would be “resetting the clock” for a number of work items already considered work in progress for some period of time, and with expectations, I’m sure they would be completed within some known SLA.
Just what I like, “real world” questions in particular when they come with some “pain points” clearly identified upfront. As I continued to think about this one more, it felt like a “perfect” opportunity to raise what I feel is a related question and discuss them together in this post. When discussing metrics as part of measuring and managing flow, the question below is one I raise often with others who are applying the Kanban Method to their workflow processes.
“In your workflow process context: Does Data Shape Your Policy or Does Policy Shape Your Data?”
How might we in the case of a “Lab Week” produce the metrics we want and visualize this information without any “discomfort” of doing post collection adjustments or moving (resetting) work items back into the backlog? What can we learn from thinking about and responding to this “Lab Week” question that applies beyond the original basis itself in how we develop policies for managing our workflow processes?
Written in direct collaboration with Richard Hensley (McKesson AVP).
“Knowledge is an unending adventure at the edge of Uncertainty.” – Jacob Bronowski, British mathematician, biologist, historian of science, poet, inventor, author of “The Ascent of Man”
“Maturity of mind is the capacity to endure Uncertainty.” – John Huston Finley, 1938 – New York Times editor-in-chief
“Many high performers would rather do the wrong thing well than do the right thing poorly.” – Thomas J. Delong & Sara DeLong, “Managing Yourself: The Paradox of Excellence”, June 2011 – Harvard Business Review
It’s common to see an organization (the people in them) focus on building systems with as many features as possible and targeting delivery by a specific due date. Yet, often the result is missing the date while also ignoring other important goals demanded by the businesses such as high levels of product quality, development productivity, planning reliability, employee satisfaction, and customer loyalty. Retrospectives, if done after such an occurrence, surface the dissatisfaction concerning missed dates, poor quality, technical debt, and more, still frequently this pattern repeats. Does this scenario sound familiar to you? If so, why do you think it is so? In a past or maybe your current organization, I’m guessing you’ve heard or thought, “We need our business and technology people on the same page.” How might “being on the same page” look in your organization? Does your current software development methodology, its principles, processes, and practices, contribute effectively to this objective? Is getting on the “same page” with objectives and goals enough?
Over the last three years Richard Hensley, AVP Process at McKesson Health Solutions, has worked to address these challenges within the three business units of the division he works in at McKesson. I’ve been able to catch up with him on several occasions and discuss his efforts over these years. This post is a brief “fly-over” of his experience, capturing some of the key thoughts Richard developed over this time and shared during our conversations.
No Silver Bullets Here!
Anyone reading a blog post on getting business and technology people on the same page probably knows there is no “silver bullet” to this challenge. Chances are good you’ve already been a part of such an effort, right? If so, you know it is not a simple task. Or even if you saw them start out on the same page in your organization, did issues appear that caused this working relationship to return to its earlier challenging state?
In short, Richard suggests, “you won’t find quick fixes for this issue” and his experience indicates it takes a serious effort from both sides. It also requires a continuous commitment over time because “getting everyone on the same page is a good start but not sufficient to sustain the business.” Yes, getting on the same page upfront, by itself, can be a positive thing as it helps everyone feel good, knowing where others are at and being in the know. If you’re familiar with the practice of daily stand-ups, you can draw a comparison, getting on the same page only is like a stand-up where team members simply provide “status” of their work. It is a good start, but alone it is ineffective for helping to solve real issues. In a similar fashion, getting on the same page with objectives and goals, if that is all you do, doesn’t help anyone address how things might be done more quickly or more effectively. It is a good start, but more is needed! (more…)
Note: My colleague Dan Vacanti has also captured and expanded in greater detail on much of the topics touched on in this post, in his book titled “ActionableAgile” which you can learn more about here.
“The ability to take data – to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it, it’s going to be a hugely important skill in the next decades, not only at the professional level but even at the educational level for elementary school kids, for high school kids, for college kids. Now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it.”
“I think statisticians are part of it, but it’s just a part. You also want to be able to visualize the data, communicate the data, and utilize it effectively. But I do think those skills – of being able to access, understand, and communicate the insights you get from data analysis – are going to be extremely important. Managers need to be able to access and understand the data themselves.”
– Hal Varian, Google’s Chief Economist, Jan 2009 – The McKinsey Quarterly
My previous post discussed how some of my earlier teams used T-shirt sizes for story level work items in their software development planning processes. But T-shirt sizes were only a part of what helped us get effectively predictable. The emphasis on just-in-time (JIT) story creation and story analysis, along with just-enough story and portfolio level backlogs (limiting work-in-progress or WIP), were also significant contributing factors. Two other key factors were de-emphasizing upfront estimating of level-of-effort and duration, and instead placing a greater emphasis on lightweight tracking of real (lead and cycle) times to complete and deliver story level work items. (See my earlier posts here and here for a bit more context and background on push vs. pull scheduling systems.)
I also discussed how some basic analysis of this lead time data and T-Shirt sizing helped us develop an internal service level of agreement (SLA) for completing story level work items. But this information also guided and shaped the policies we developed to influence the team’s interactions and specific responses (pulled from our toolbox) in a JIT manner as information unfolded about a story’s level of effort and duration. My observation is that all this contributed to us becoming predictable in a context where we had never been before using heavier upfront planning strategies. Based on my study of scheduling systems this combination reflected key pull scheduling characteristics, where the role of our software development workflow management changed from determining all operation activities upfront to one focused much more on setting the rules for interactions (in turn influencing our work environment structures).
There’s lots more analysis (mathematical and statistical) you can do using the minimal and easily collected data that produced the Basic Story (Lead Time) Metrics table and T-shirt sizes from my earlier post and that will have to wait for another time. For this post, I want to focus on visualizing the information in this simple table to see if we might extract a bit more value from the modest analysis investment already expended.
I’ll also build on this initial visualization, using a bit more (quick) low hanging fruit type analysis of the full raw data set represented by the earlier spreadsheet snippet, to produce a basic temporal (time) perspective of story lead times. Can adding a basic temporal perspective provide a number of other useful insights into understanding the nature of our workflow’s story level work item lead times? (Both the earlier table and spreadsheet snippet are included here; click images to enlarge). To this basic temporal perspective, with a bit more new analysis on the existing raw data spreadsheet, I’ll then add new information extracted to give a work item type perspective as well, and then overlay the T-shirt size information to this mix.
Finally, again with just a bit more analysis on the existing raw data, I’ll visualize two other separate temporal perspectives using both percentages and frequency counts. Afterward, let me know what you think. Do one or two of these other ways of visualizing the information in the earlier table and spreadsheet snippets help you and others access, understand, and communicate insights that leads to a more effective predictable workflow in your software development context? (more…)
Note: Post updated on 3/29/12 to correct titles on the three T-shirt size tables and to lighten the blue shade on the Medium T-shirt size row in these tables to make it easier to read.
“We do estimation very badly in the software field. Most of our estimates are more like wishes than realistic targets.”
– Robert L. Glass, Nov 2002 – Facts and Fallacies of Software Engineering
“The best way to get a project done faster is to start sooner.”
– Jim Highsmith
In earlier posts (here, here, and here), I commented briefly about moving away from upfront estimating and toward tracking and measuring “actual” metrics to complete work items and using this data as a foundation for predictable scheduling and forecasting. Am I suggesting your software development team can’t achieve effective “predictable” scheduling using upfront estimating? No. But first, to be clear, I’m delineating upfront estimating activities from upfront analysis activities. The former is associated with predicting the number of calendar days (duration) to complete a work item and the actual hours (effort) expended on a work item. The latter is associated more with partitioning larger work items (epic, feature) into smaller ones (story, task) and determining relationships and dependencies between them to assess complexity or risk.1 The significance for decoupling these becomes apparent shortly. So, in your context, if teams haven’t achieved effective predictable scheduling using upfront estimating, I am suggesting it might be beneficial to approach project planning from what is likely a different perspective. I’m also suggesting that over time this different perspective may reduce planning effort.
As an example, I’ll focus on one of the “actual” measurements my past teams collected, in this case, Lead times for “story” work items, and discuss how we used this data to derive a very predictable range of “T-shirt” sizes. This range helped us predictably manage story level work items thru our workflow, and enabled us to identify issues earlier as they emerged for those needing alternative (special) processing or management. Before moving to how and what we did and the benefits obtained, I’ll briefly provide context for why we shifted perspective, which may reveal some insights for benefit in your environment.
Earlier Times in a Nutshell
Our development teams were well into implementing agile processes and demonstrated visible improvements in several areas related to technical practices and our ability to deliver more frequently. However, we continually missed “commitments” for the number of stories to be completed in a sprint whether it was two weeks or 30 days. We were unsuccessful at effectively predicting duration or effort for completing stories using upfront estimating techniques that we had tried including variations of planning poker with ideal days, points, hours, Fibonacci sequences, etc. I won’t go into consequences of these estimating efforts here but will highlight a key observation made during efforts to mitigate these costs that led to shifting our planning perspective.
As the mid-sprint approached and we saw our commitment at risk, at one point a team decided to assertively reduce (“de-commit”) work items (stories in this case) in their “Doing” state, moving them back to ToDo or to the story backlog. Again and again, when we did this in a sprint, stories remaining in the Doing state would get completed quickly, and then we’d “whip back” the other way and re-pull stories before the sprint end. After of a few of these “herky-jerky” sprints, we did something different and shifted to a “pull-scheduling” perspective.
Push vs Pull Scheduling?
This image is my summary derived from sources contrasting push and pull scheduling systems.2 Push scheduling starts with obtaining commitments to upfront estimates of duration and effort, then develops a schedule based on these estimates and seeks predictable outcomes by attempting to adhere to the developed schedule. Pull scheduling starts with obtaining predictability of upfront (measured) capabilities and known constraints, then develops a schedule based on these capabilities and constraints and seeks to achieve commitments to the developed schedule. Is this distinction a bit too intangible for you? (more…)