There is a common perception within I.T. that Agile methods are recent innovations - the new kids on the block - and they are contrasted with the traditional waterfall approach - the old-timer that has been around for ages. This perception is propagated by events such as the widely-discussed 10-year anniversary of the agile manifesto and by the ongoing challenge of the "old guard" - either publicly or within organizations - of the effectiveness of agile versus waterfall. I recently read two articles, however, that convincingly shatter these misperceptions and lay bare the shocking truth.
While the term agile itself is indeed ten years old, the philosophy and approach of iterative and incremental development (IID) has a surprisingly rich and extensive history. The article Iterative and Incremental Development: A Brief History (PDF) by Craig Larman and Victor R. Basili published in IEEE Computer discusses how the ideas of IID actually predate the existence of software, and have been used and promoted in every decade since for over 50 years. This article also discusses a classic 1970 article by Winston Royce well-known for supposedly promoting the use of waterfall, but which actually recommended the development of an initial pilot or preliminary version prior to creating the final version intended for delivery to the client. Larman's and Basili's article also discusses the ongoing evolution of the standard used by the U.S. Department of Defense for software development. The initial standard was document-heavy, gated, single-pass waterfall, but the high rate of project failures led to first the allowance of and eventually the full adoption of IID approaches.
So a limited set of organizations (often governments due to their byzantine contracting restrictions) did experience an evolution from waterfall to agile over time, but notably they started with a misunderstood version of waterfall. The use of IID approaches has always been part of the I.T. industry.
What about the effectiveness of agile? The second article Quality metrics: Software quality attributes and their rankings is an interview with Capers Jones and Olivier Bonsignour regarding their new book The Economics of Software Quality. The authors in this book discuss the effectiveness of over 100 quality factors using research based on over 10,000 software projects. On a scale of -10 for extremely harmful to a scale of +10 for extremely valuable, Agile methods rated a 9 - highly valuable - while waterfall only rated a 1 - barely useful.
These two articles highlight the fact that agile-like methods have been in use since the start of the software field, and are on average far more effective than the waterfall approach. This leads me to conclude that there is really no defensible reason for organizations to mandate or promote the use of waterfall.
If you find this article helpful, please make a donation.