«    »

My Defect Fixing Process

What's your process for fixing a defect? What do you do when you are informed that a feature you developed isn't working to the users' satisfaction, or even worse fails to work at all? Here's what I do.

  1. Initial investigation. My goal is to reproduce the reported problem in the application in my development environment. This may require obtaining more information from the user(s). If I can't reproduce it, I try to reason about possible causes to help me figure out what conditions are necessary to reproduce it. In hard cases, I may need to add extra error-checking or error-handling code to try to narrow down what is going on when the problem happens again.
  2. Classify the problem. Many problems are defects: a flaw in the code or design that prevents the application from working as designed. Sometimes the user may be having troubles with the user interface - I classify this as a usability defect. Or the existing functionality may be too limited and the user is requesting something new - this I classify as an enhancement request. The distinction between defect, usability defect and enhancement is quite blurry. Often I will treat a usability defect like any other defect that needs to be fixed, but if such a defect requires significant user interface changes, then I classify it as an enhancement. If I determine that the reported problem is really an enhancement request, then I am finished with my defect fixing process (enhancements are typically handled through a different process).
  3. Analyze the cause. My goal is to write an automated unit test that proves the existence of the defect by failing - i.e. I'd expect the test to pass if the defect didn't exist. I may need to do some analysis to determine what section of code is failing, and may need to iterate on this a few times if the defect is especially hard to pin down. For certain problems, including most usability defects, it is too difficult to write an effective test so I rely on manual testing and debugging.
  4. Fix the defect. Now that I know the cause of the defect, I can fix it. This is where having a test for the defect really pays off, since I can run the tests after making my fix and verify that the problem is corrected by having the tests pass.
  5. Learn from the defect. The defect may be fixed, but I'm not done. Each defect is a learning opportunity; it represents a failure in something I've done that I can correct and improve. So I ask myself questions like the following. Why or how was this defect introduced into the code? Was it an error in design, a copy-and-paste error, a failure to consider special conditions like a null return value, improperly understood library, etc.? Why didn't the unit tests catch this problem? Why didn't reviews or manual testing catch this problem? Would similar defects exist elsewhere in the code, due to the same or a related failure that caused this problem? What can I do to prevent this type of defect from happening again?
  6. Act on my learning. Based on what I've learned from this defect, I take action to improve myself, the application, and the team's processes for the future. Possible actions include writing more unit tests for the problematic section of code, refactoring a difficult-to-use API to eliminate the problem that occurred, informing other developers of why this defect happened, and improving error handling to make it easier to reproduce similar defects in the future.

If you find this article helpful, please make a donation.

4 Comments on “My Defect Fixing Process”

  1. Mike Totman says:

    One other thing that you could put in point #6 “Act on my learning” : Look for other occurences of that error. If that error is an easily searchable pattern or commonly occurring idiom, search out other cases of similar code and see if they suffer from the same defect.

    This is the approach the OpenBSD team applies to eradicate whole families of defects which lead to vulnerabilities. e.g. buffer overflows.

  2. I agree – looking for other occurances is a very useful thing to do. I did mention this in point #5 “Learn from the defect”, but perhaps it should have gone under point #6.

  3. Chris Wash says:

    I like this process.

    W/R/T Mike’s comment, IntelliJ’s structural search (& replace) makes this idea of finding recurrences quite compelling. See: http://www.jetbrains.com/idea/documentation/ssr.html

    Also, from what I remember Chapter 4.6 of “Test Driven” by Lasse Koskela is another good resource for activities involved in making maintenance changes (applied specifically to legacy code). Link:
    http://www.manning.com/koskela/

  4. Abhishek says:

    Regression testing should be included in the context

«    »