Working as a software developer offers many challenges ranging from tracking down an elusive defect in third-party software to trying to determine what a client really wants. Perhaps the most difficult challenge, however, is succeeding as an oracle who can accurately divine the future – more commonly referred to as creating an estimate. A layperson (or worse, a manager :) might think that producing a software estimate is fairly straightforward – like asking for an estimate for a trade (i.e. a plumber or electrician) to do some work in your home. Construction projects, where estimates are the norm, are often used as an analogy or comparable for software development. So what is different about software development that makes estimating so difficult?
I have already alluded to an essential concept: creating an estimate is attempting to predict the future. The basis of all estimation techniques is to extrapolate the future based on work done in the past, so the higher the likelihood that the future matches the past, the more accurate the estimate. How well the future matches the past is determined by a number of factors:
- How well can you predict what work must be done in the future? In other words, how clearly do you understand the scope and requirements for the activity you need to estimate? I do not feel that software development is much different in this regard from other fields of endeavor: whenever you ask a trade for an estimate, they first ask questions to determine the size of the job to be done. For construction projects, there are blueprints that define the scope of what needs to be built. Even a well-defined requirements specification, unfortunately, is not enough, even ignoring the likely scenario of changing requirements. Within software development, requirements do not actually define the work that needs to be done, just how the system is to function. It is the architecture and design of the system that determines the work necessary to construct it. While a high level design (i.e. architecture) can be done initially, this is still insufficient for an accurate estimate. A detailed or low-level design is necessary to clearly identify the work to be done, and will typically need revisions throughout construction (assuming it is done up front).
- How similar is the work to be estimated to work done in the past? The more change from before, the more difficult it will be to estimate. This is why research or discovery types of activities are essentially impossible to estimate: if it has never been done before, there is no real basis for an estimate. Activities involving lots of thinking and creativity are similar. Significant portions of software development projects fall into this category, including requirements analysis, design, and defect fixing. I am a proponent of the viewpoint that coding is itself often a creative design activity: writing code is the final step in specifying the design of the system. (For an excellent argument in favor of this viewpoint, check out the article What is Software Design?.) Software development also experiences a rapid rate of change in terms of tools and technology: the work that needs to be done might be similar to that done five years ago, but if it is using new technology then an effective estimate is much more difficult.
- Who will be performing the work? It is difficult enough to estimate how long a software development activity will take for yourself, let along for someone else whose skill level and experience you have no idea of. This is particularly significant given that software engineering research has identified that developer productivity can vary by as much as an order of magnitude - 10 times difference between the best and worst, and 2 times difference between the top and bottom halves (See the book Peopleware: Productive Projects and Teams starting at page 44 for more details.)
Making an accurate estimate is difficult. I define estimate accuracy as the percentage difference between the estimated and actual value. Lower numbers are better. For example, an estimate of 4 months that actually took 6 months had an accuracy (difference) of 50%. An estimate of 4 months that took 4.5 months was much more accurate at 12.5% difference. This implies that the actual accuracy of an estimate can only be evaluated when the activity is completed. Estimates can (and should) include a predicted accuracy, such as 4 months +/- 50%, but note that this predicated accuracy is itself an estimate. One of my favorite flippant responses when asked for an estimate is to say it will take less than two years. I haven't been wrong yet :) Unfortunately, like predictions of the future, estimates of such low accuracy hold no value.
Management typically insists on accurate estimates in order to have firm schedules, cost projections, and resource utilization plans. In the worst case I have seen there was a service level agreement requiring an estimate accuracy of plus or minus five percent. I consider requiring such accurate estimates to be unreasonable for the simple reason that the more accurate the estimate must be, the more effort must be expended to create that estimate. And the only effective way to significantly improve the accuracy of an estimate is to proceed with the activities or project to be estimated. Steve McConnell does an excellent job explaining this in his book Rapid Development, starting at page 165. Essentially, a project starts out with a fuzzy conception of what needs to be done, so any estimate done in this phase is likewise fuzzy. As the project proceeds and the final product becomes clearer, the estimate can become more accurate. Research quoted by Steve indicates estimate accuracies are around 400% at initial product concept, 50% after requirements, and fall to 10% after detailed design is completed. To achieve that 5% estimate accuracy would likely require over half of the coding to be completed.
Lack of awareness about the difficulty of estimating leads to unrealistic expectations and demands. This in turn saddles projects with unrealistic budgets or schedules from day one, setting them up for failure. I believe the I.T. industry would benefit greatly from an increased understanding of estimation and its inherent difficulty. Realizing that an estimate is nothing more than a prediction of the future whose accuracy depends on how clearly our understanding of the future matches how it unfolds is the first step.
If you find this article helpful, please make a donation.