«    »

Cyclic Incremental Growth in Software Systems

I recently read an interesting research paper titled Metrics and Laws of Software Evolution - The Nineties View by Lehman et al in the 1997 proceedings of the Software Metrics Symposium that discusses eight laws of software evolution. One set of graphs within the paper jumped out at me: they show the incremental growth of two systems over a series of releases. The graphs shared a common pattern: releases with higher than average incremental growth tended to alternate with one or more releases with lower than average growth. What are the implications of this commonly observed behavior for developing and using software? How can we use this pattern to our advantage? Read on to see my answers to these questions.

The following graph is illustrative of the pattern:
Incremental Growth of Software by Release

Cause of Cyclical Incremental Growth

Lehman's general interpretation of these cycles was based on systems thinking in terms of feedback loops. Positive feedback – such as user requests for enhancements – leads to incremental growth in the next release. This in turn activates forces causing negative feedback – such as an increased number of defects to fix – which brings the growth back down. The following diagram illustrates these feedback loops causing a limit to growth:
Enhancements and Fixes Feedback Diagram

If you are not familiar with the topic of systems thinking then I recommend the book The Fifth Discipline Fieldbook, which provides an insightful introduction to the topic of systems thinking and the use of these feedback diagrams.

Lehman discussed briefly possible causes of the cycles but did not explore this topic in depth. Based on my experience I believe that one primary cause of such cycles is that after significant enhancements to a system are released and put to use, defects, operational issues, and desirable minor changes are discovered. The next release(s) then focuses on addressing these issues, which leads to much lower incremental growth in the system. Once these issues are addressed, the subsequent release can focus more on adding features, which results in increased incremental growth and leads to the observed pattern of cyclic incremental growth.

I have observed this pattern occur multiple times with software I am involved in maintaining. One component at work I have been involved with for a few years now and consistently after every release with significant changes or additions to functionality there is a subsequent release to address defects, operational issues, or usability issues discovered through daily use of the component in production. I have seen a similar pattern with the open source software I distribute on my website. Often after releasing a new feature and using the software, I discover either a defect or a usability issue which I then fix and release a new version.

Preventing the Cycles

Can these cycles be prevented? In particular, can these issues found after the release goes to production be discovered instead through testing the system before release? I have no doubt that thorough testing would help reduce the number of issues found post-release. Extra testing will not, however, prevent the cycle. Software engineering research tells us that testing only finds a certain percentage of defects - usually under 50%. When I consider from my experience the types of issues discovered post-release, they fall into one of the following categories:

  • Usability: The system functions as per the requirements and design, but is difficult or confusing for real users to use in practice.
  • Maintenance issue: Production support staff experience difficulties supporting the operation of the application in production, i.e. due to insufficient logging or configuration options, or improper error handling.
  • Incorrect requirements: Once users actually use the system, they discover that what they asked for (the requirements) is not exactly what they want.
  • High-volume usage issues: I include in this category various types of issues that only seem to be discovered in production use including concurrency, compatibility, performance, and scalability.
  • Functional Defect: A failure of the system to function as per the functional requirements and design.

While I would expect most functional defects to be caught prior to release with careful testing, most of the other categories are less likely. To even have a reasonable chance of discovering these other issues requires a special focus on these categories during development and testing – i.e. including usability testing or stress testing. Given that functional testing is frequently cut short to meet project deadlines, these other types of testing are much less likely to be done at all, let alone thoroughly.

The fundamental issue is that the most thorough way to test software is to put it into active, daily use. (This is not necessarily the most cost effective way nor is it best for the users.) So despite all the testing that may be done prior to release, deploying the software to production and letting users hammer it is going to reveal issues. I believe that this leads, inevitably, to the observed cycles of incremental growth, as one or more follow-up releases are needed to address defects. The amplitude and frequency of the cycles can be reduced, but not eliminated.

Lessons Learned

What lessons can be learned from this pattern of cyclical incremental growth? What common practices in I.T. try to address this cycle?

  • For risk-adverse organizations, avoid software that has just been released with significant enhancements – i.e. release X.0. Instead wait for subsequent minor releases that address the inevitable issues found after the release – i.e. release X.0.1 or X.1.
  • Use beta releases to simulate an actual release. Whether the beta testers are a handpicked subset of existing users or are early adopters who voluntarily download the beta, the idea is to put the software into hands of users with the agreed expectation that there will be issues. This needs to balance two competing forces: putting the software to use as widely as possible to find issues versus keeping users happy about the product – which finding defects makes more difficult.
  • Software development teams should plan for an issue-fixing release after major enhancement releases, especially if other steps are not taken to minimize the cyclical pattern. This is surprisingly difficult as it is far easier for the team and management to stay in denial ("the software has no defects" being a common refrain of developers) than to accept reality and plan accordingly.

If you find this article helpful, please make a donation.

«    »