«    »

Defects – To Fix or Not to Fix

To fix defects or not fix defects, that is the question: whether it is better to suffer the complaints of outraged users, or to divert effort to investigate and eliminate them.

Shakespeare quotes aside, every software development project has to make decisions on how many defects to fix and which ones to leave alone prior to shipping. While I have seldom seen this question debated within projects, the advice from industry thought leaders varies considerably. The Agile and Lean methods of software development in particular have somewhat opposing perspectives.

I believe that considering both sides of this question provides a fuller understanding of the issues and better equips us to answer appropriately. Therefore in the two sections below I explore the reasons behind both sides of the debate.

To Fix

  • Shipping poor quality, defect-ridden code can upset users, turn away customers, and lead to a hard-to-shake bad reputation.
  • The decision that a feature is worth developing is made with the expectation that it will work correctly. So any defects found in a feature means that the feature is still incomplete until these issues are fixed.
  • Defects provide feedback regarding the development process. Each defect represents an opportunity to do a root cause analysis of what led to the defect and put countermeasures in place to prevent re-occurrence. The Lean mindset of "Stop the line" demands that new development be put on hold to fix newly discovered defects.
  • Defects introduce the risk of compounding quality problems. The impact of a defect can be more significant than initially realized. Defects can be inadvertently replicated in other parts of the system. Enhancing components with too many defects can slow progress to a halt, as the system becomes essentially a shifting quicksand that is too unstable to work on. Constantly fixing defects helps maintain a high velocity of development over time.
  • To mitigate risks in not fixing defects, each defect needs to be analyzed to understand its impact, cause, and required changes to fix. But after performing this analysis most of the work is usually done - the fix is relatively straightforward. Waiting to decide later to fix the defect (e.g. in a subsequent release) causes all the knowledge gained in the analysis to decay over time which is wasteful (in the Lean sense).

Not To Fix

  • Significantly delaying the release of software to fix all defects leads to a loss of immediate revenue and potentially loss of market share due to competitors beating you to market. So you cannot afford to wait to fix all defects.
  • The entrepreneurial mindset, especially for startups, is to ship early to get feedback from paying customers. Perfection is the enemy of the good.
  • Under at least some versions of Scrum, defects are considered new tasks that are added to the product backlog to be prioritized by the product owner. This prioritization is based on the defect's impact (severity and likelihood of occurrence) and the effort required to fix it. Many minor defects will therefore likely never be fixed as new functionality will typically be of higher value.
  • Stopping to analyze and fix defects disrupts developers who are in the middle of working on other functionality and is wasteful.
  • Fixing defects in functionality that is already otherwise finished development and testing will require additional regression testing. Not fixing now and waiting until enhancements to this functionality are needed minimizes the extra effort required.

Conclusion

Shakespeare was wrong. There is actually a third perspective regarding whether or not to fix defects: avoid the question as much as possible by focusing on defect prevention. The Lean mindset of building quality in avoids all the waste associated with finding, analyzing, and fixing defects and should be our preferred approach. Only when it fails and the occasional defect is introduced do we then have to answer the question.

If you find this article helpful, please make a donation.

2 Comments on “Defects – To Fix or Not to Fix”

  1. Ben Linders says:

    Fully agree that you should ask yourself whether or not to fix defects. Additionally you can ask yourself which defects that you don’t want to happen again, and use Root Cause Analysis to take preventive actions.

    There is a lot of discussion on whether or not to add defects to the backlog. Either way, the priority of solving them should be balanced with ongoing development. For significant problems, the business needs and thus priority will be high enough to include them into the sprint, and get them solved!

  2. [...] Extended with graphs).Related articlesImproving Data Quality Through Anomaly Detection (sei.cmu.edu)Defects – To Fix or Not to Fix (basilv.com)Related posts:Real QA ManifestoDoctor, what’s wrong with my software?Business [...]

«    »