Tooling

WebSphere provides the wsadmin command-line interface for interfacing with WebSphere servers. Two scripting languages can be used: JACL (the default) and Jython, a Java-based version of Python. Invoke wsadmin with the argument "-lang jython" to use Jython. I recommend the use of Jython for two reasons. First, it is a more mainstream language. Second, the WebSphere Admin Console shows scripting commands in Jython equivalent to the GUI operations you last performed. See the screenshot below for an example:

You may think that since the Admin Console provides equivalent scripting commands to GUI actions that you could just copy and paste these commands into a script and you would be set. Unfortunately if you want the script to be re-runnable (on a given server) or reusable (on other servers), you will almost always need to make modifications. Certain operations, like creating a data source, will fail if there is an existing data source with the same name. So to have a re-runnable script you will need to add logic to your script to first detect if the data source you want to create already exists. Many commands use a hard-coded configuration id to refer to a particular item to be configured. This id generally consists of the item's name, directory path to the configuration file, name of the XML configuration file, and XML ID of the item within the file. While the first three are constant, the XML ID appears to be a numerically generated ID that will vary between servers. So you need to change the script to look up the configuration id.

The changes required to adapt the script for broader use are not trivial and require use of WebSphere APIs - in particular the AdminConfig and AdminTask objects provided to Jython scripts that are executed within wsadmin. The best tooling I have found for writing these scripts is to use Rational Application Developer which includes a Jython editor that includes wsadmin objects as part of the content assist. See the screenshot below for an example:

Examples

With the tooling in place let us look at some examples. The following scripts all assume a simple WebSphere topology of a single node with a single server that we are setting up - no clusters or multiple servers to worry about.
A number of settings require the configuration id or name of the server or node, so here's how to look them up:

nodeConfigId = AdminConfig.getid('/Node:/')
nodeName = AdminConfig.showAttribute(nodeConfigId, 'name')
serverConfigId = AdminConfig.getid('/Server:/')
serverName = AdminConfig.showAttribute(serverConfigId, 'name')

AdminConfig.getid() is one of the primary methods to use to look up configuration ids. But what is that "/Server:/" syntax used as an argument? That is called a containment path, which is a XPath-like syntax for looking up configuration ids. For more important details on containment paths and configuration ids read this excellent article by Vincent Partington.
As mentioned above, this only works when there is a single server defined on the node - with multiple servers, the call AdminConfig.getid('/Server:/') would return multiple server config ids separated by newlines. In this case if you want to configure a specific server you can look up the server configuration id by name as follows:

serverName = "testserver1"
serverConfigId = AdminConfig.getid('/Server:' + serverName + '/')

Having a configuration id is a start, but we still have not changed any settings. So here's a basic example of setting the maximum heap size to 1 GB:

maxHeapMb = 1024
jvmConfigId=AdminConfig.getid('/JavaVirtualMachine:/')
AdminConfig.modify(jvmConfigId, '[ [maximumHeapSize "' + maxHeapMb + '"]  ]')

This follows the standard pattern of first obtaining the configuration id and then changing the setting. I obtained the code for the AdminConfig.modify call from the Admin Console. But how did I figure out the AdminConfig call, in particular which containment path to use? This was painful initially to figure out. The hard-coded configuration id provided within the AdminConfig.modify call was "(cells/Node1Cell/nodes/Node1/servers/Server1|server.xml#JavaVirtualMachine_1183121908656)". The prefix "JavaVirtualMachine" on the XML id is the key type to use in the containment path.

Not all configuration commands require a configuration id. The AdminTask object typically takes arguments specifying the server and node name. Here's an example that prevents applications from accessing internal WebSphere classes:

AdminTask.setJVMProperties('[-serverName ' + serverName +
' -nodeName ' + nodeName + ' -internalClassAccessMode RESTRICT]')

The last step in any configuration script is to save the configuration changes. This is straightforward:

AdminConfig.save()

Resources

Here are some useful resources for this topic:

• Excellent article on the differences between configuration ids, containment paths and object names for WebSphere scripting.
• IBM article introducing scripting WebSphere using Jython
• Documentation on the WebSphere 7 Express sample scripts library
• List of commands available on the AdminConfig object
• Free online Jython language book
• Book published by IBM called "WebSphere Application Server Administration Using Jython"

Happy scripting!

This chain of thought then prompted me to ask myself this same question. What do I find most disturbing when reviewing code? My instinctive reaction was to answer "defects", but upon a little reflection I realized this was not true - I expect to find defects as a fundamental principle of quality. So it usually does not bother me to find a few during a review.

There are times when I am very disappointed when reviewing code - what am I finding at those times? Here are some specific occurrences:

• Code riddled with defects reflecting a fundamental lack of understanding about the requirements.
• Code very difficult to understand due to poor names and overly complicated logic that seemed repetitive or unnecessary.
• A large amount of non-GUI code written without any supporting tests.
• Code with inconsistent formatting and style.

What is the common theme here? After further reflection, I realized that the common element underlying these situations that I find most disturbing is a lack of professionalism / craftsmanship. This can manifest in a number of ways as indicated by the above list. The key criteria is whether a developer is helping achieve or is hindering our mission as software developers, based on what they produce for code. My evaluation of what is most disturbing is at its essence based on my core values and beliefs concerning software development.

Too many developers think their job is complete once a feature has been coded. Sometimes they think that it is the tester’s job to find defects. Sometimes they think defects in released code are unavoidable and normal, so not worth worrying about. Sometimes they believe their code is perfect - it cannot possibly have defects. I encounter developers with these attitudes with unfortunate frequency. I also encounter development managers who are surprised to encounter such attitudes. A while back I talked to one manager who was shocked to learn than one group of developers under her were assuming their code worked if it compiled successfully - there were no reviews or any sort of testing being done. So I hope with this article to raise the awareness amongst developers that coding is simply not enough to produce working software, and to raise the awareness amongst development managers that they need to ensure the appropriate systems are in place to support this.

The reality is that even the most diligent developers inject defects into their code at a surprisingly high rate. Defect rates are often defined as the ratio of the number of defects per one thousand lines of code (KLOC). Industry statistics on defect rates are rather hard to find and vary significantly, partly because the definition of defect used varies. Several studies have reported defect rates in the range of 10 to 100 defects per KLOC as reported in the book Best Kept Secrets of Peer Code Review. This works out to one defect per 10 to 100 lines of code.

On my most recent project I decided to calculate the defect rate for a particularly error-prone feature. Counting only defects found by independent testers after code reviews and unit testing were done, and using a KLOC count not including comments or blank lines, this feature had 20 defects for roughly 850 lines of code which is a defect rate of 24 defects per KLOC, or one defect for every 40 lines of code. This may seem reasonable, but remember that this is after multiple code reviews and automated unit testing have already found and eliminated a number of defects. (How many I do not know as these kinds of defects are not tracked.) And there still may be yet-to-be-found defects still lurking in this code. So the actual defect injection ratio is higher, perhaps much higher.

Defect rates have such a wide variance, even between developers working on the same code base, that it is unfortunately not a reliable metric for predicting defect counts. My main point in discussing them is to emphasize just how frequently defects are introduced.

Coding, therefore, is simply not enough. Every developer needs to have a personal system for preventing and eliminating defects, which should integrate into the system / processes used by the development team to produce high-quality working software. For ideas on how to assemble such a system check out my definition of done that identifies a number of defect elimination activities.

The idea of to do comments really only makes sense with the proper tooling. The Eclipse integrated development environment calls them task tags, and supports providing any number of custom identifiers that when found in a comment will cause Eclipse to add that comment into its Task view. The default identifiers that Eclipse ships with include "FIXME" and "TODO". You can then browse the task view, sorting or filtering the tasks by various criteria, to see the outstanding work. Continuous integration servers such as Hudson running the Task Scanner plugin can produce statistics, reports, and graphs of the outstanding tasks in a code base.

Why use to do comments?

When doing my own coding, I use to do comments when ideas come to me regarding code I need to write or special cases I need to handle that are separate from what I am coding right now (usually in order to get the current test to pass, via test-driven development). Writing the idea down gets it out of my mind and out of my way. Using a to do tag reassures me that it will not be forgotten: part of my definition of done is to ensure that there are no to do comments remaining in the code.

When looking at code written by other team members, I want to be able to quickly tell what state the code is in – is it completely finished, or is it a work-in-progress, with scenarios or requirements left to be handled? The reason I care is that if I think the code is supposed to be finished, and see issues or outstanding work, then I will raise the issue(s) with the developer. Fairly often when I have done this in the past, the developer reassures me that they were already aware of the issue and would resolve it. That is usually when I recommend the use of to do tags as a communications mechanism to the rest of the team as to the status of the code, especially if someone else had to take over working on that code. And as often as not, unrecorded issues that developers say they are aware of and will resolve later end up being forgotten and left unresolved.

I hope I have convinced you of the value of using to do comments. Please leave a comment and let me know what you think about the practice.

This rule is based on a bottom-up engineering philosophy: if you completely understand the methods you are invoking, then you should understand the behavior of the method you are coding and know that it will work. This applies recursively up the call stack to the top-level entry point of your application.

Key Requirements

My formulation of the rule above specifies two key requirements for using another method:

1. Behavior Understood: The behavior of a method can be defined in terms of its pre-conditions and post-conditions. Knowing the pre-conditions allows you to ensure you are correctly invoking the method, while knowing the post-conditions ensures that you will get the results you want.
2. Confident Will Work: This is arguably part of the prior requirement - understanding a method’s actual (rather than stated or expected) post-conditions - but it is so important I felt it should be explicitly stated. You need to verify that the method you are using will actually function correctly and not fail due to a defect. This verification is best done via automated tests, but there are scenarios I discuss below when another approach is needed.

Applying the Rule

Applying the Use Understood Methods Rule involves, therefore, satisfying these two requirements. Exactly how I do this varies based on the context – specifically the nature of the method I intend to use. Below I describe the common scenarios I encounter and the approach I use to apply the rule to each.

• Method with implementation in code base: Calling a method that has an existing implementation within the code base you are working on is the most common scenario. This can be a method on the same class, on a super class, on a collaborating class, or a static method. This also can be an abstract method defined on an interface or abstract base class for which the implementing class exists and is known. To understand the method’s behavior I refer to the documented pre- and post- conditions if available, otherwise I look at the source code for the method. The correctness of the method should already be verified through automated tests. If necessary I can use code coverage analysis results to confirm that this method has sufficient test coverage.

• Method to be written concurrently in code base: Often when coding a method, I find I have to create a new method, either on the same class or on a separate class. This might be a simple extract-method refactoring of logic to simplify the existing method, or it might be new functionality required on another class. In this scenario I have no issues understanding the behavior of the new method as I am writing it at the same time. If the new method is non-trivial then I ensure it works by creating unit tests for it separate from my tests for the original method I am working on.

• Method with unknown implementation in code base: This applies to abstract methods in interfaces and abstract base classes for which no implementation yet exists or for which the implementation cannot be known in the context of the method being written. This latter scenario is typical when there are multiple implementations being processed in a common fashion. In this case I insist on having documented pre- and post- conditions for the method being invoked.

  If the method implementation does not yet exist then it is obviously not possible to verify now that it is correct, but it is possible to take steps to gain confidence that the implementation will work once it is written. One option is to ensure that the automated tests for the method you are writing that invokes this not-yet-implemented method sufficiently exercises the functionality of this not-yet-implemented method to ensure it will meet your needs. Another more general option is to ensure that the team has processes in place such as test driven development and code reviews to ensure that code written in the future will indeed work.

• Method in third-party code: When using third-party methods I usually rely on the documented API. When this is inadequate I look at the source code if it is available (this is a key benefit of open source libraries – the code is always available). In some cases the third-party software implements a specification (such as the various Java EE specs) and the specification can be used to understand what the third-party code is supposed to do.

  Once third-party software has been selected for use within a project I generally assume it works. The verification of the quality of such software happens previously, in the selection process, when I do my due diligence to evaluate the quality of the software under consideration.

  On rare occasions I write a unit test to understand and/or verify how some third-party functionality works. This is something I would probably benefit from doing more often.

Given the prevalent use of automated tests to verify correct behavior, you might be wondering how the application of this rule is impacted by the use of test driven development (TDD). The short answer is there is no impact: this rule applies the same whether or not TDD is being used. I do, however, slightly revise how I do TDD in order to apply this rule for the scenario Method to be written concurrently in code base - I create a second failing test before creating the new method. For further details see my article describing this refinement to TDD.

The Broader Context

Following the Use Understood Methods Rule is necessary for creating high quality code that you would trust your life to, but is not sufficient. Correctness also depends on satisfying class and application-wide invariants (such as properly closing database connections or limiting the number of file handles consumed concurrently), understanding the behavior of containers and frameworks (such as the Spring application context or Java EE container), and considering other quality attributes (such as security and scalability) which are and emergent in nature and not easily analyzed by looking at methods independently. I consider my rule to be the foundation on which the code is written, which these more global concepts rest on top of.

Benefits

This section covers the benefits, as I see them, of doing TDD. This does not include the benefits of doing automated unit testing, which I am a big fan of and have been doing for years using a non-TDD approach (i.e. writing tests after writing production code).

1. Using TDD provides great code coverage, especially conditional code coverage. Strictly following Robert C. Martin’s three rules of TDD should result in 100% coverage for both statements and conditionals. I allow myself to deviate from these rules at times, but still obtain 90+% coverage.

   Previously when writing tests after coding, as part of my process to ensure I was done a feature I would check the code coverage results to identify gaps and add missing tests. I have found that this code coverage check is mostly a formality when doing TDD.

2. TDD helps avoids the tedium that I have experienced at times writing the tests after coding. Often while doing TDD I am able to establish what I call the red-green rhythm: write a failing test (unit test result bar goes red), then get it to pass (bar changes to green). This rhythm makes it more enjoyable to write the tests, although the tedium is not completely eliminated. As a result, I find it takes less discipline to write the tests first than afterwards – I do not have to force myself to write a bunch of tests after getting some functionality in place.
3. When I started using TDD I was initially uncomfortable with writing the simplest possible code to get a failing test to pass (TDD rule #3) when I knew that the final production code would be different. As my experience using TDD increased, I began to see a number of benefits of following this rule:
   • Writing more code than the minimum necessary to pass the test runs the risk of having logic in the production code (statements or conditions) not covered by the tests.
   • At times, when the design for the method / class was a little fuzzy to me, writing the simplest possible code actually proved helpful, as I could then write another failing test which then clarified for me what the design would need to be.
4. Using TDD, especially when strictly following the three rules, seems to eliminate the question / debate about how much to test. When introducing unit testing to developers (not TDD, just the use of automated unit tests), I get asked this question time and time again. I have a standard answer I use, but it no longer seems relevant when doing strict TDD. In fact, the question itself no longer applies when doing TDD.

Limitations

When adopting a new practice it is important to know the contexts in which the practice is less applicable. I have come across a number of situations in which TDD seemed less helpful. These were the situations when I was most likely to deviate from the three rules of TDD or abandon TDD entirely.

1. I prefer to have the design of the method / class I am working on fairly clear in my head before I start writing tests. Sometimes I do this on paper, but sometimes I do this by working with the actual code which would be a deviation from TDD. Since adopting TDD I have tried to do this design in the test code instead of in production code, with mixed results. Sometimes this worked fine, and sometimes it felt unnatural and less productive. I have the feeling that as I gain experience with TDD I will grow more comfortable with doing design within test code.
2. When I need to code a non-trivial algorithm I often extract logic into separate methods on the same class or need to invoke new methods on collaborating classes. Often I need to write these additional methods as the minimum needed to get my failing unit test to pass, which means that I am strictly following TDD. The issue is that my normal unit testing practice is to test methods individually as much as possible, especially methods on other classes, and the rules of TDD do not require me to do so. In essence, my original failing unit test ends up being an integration-style test for the overall algorithm, while I want to have individual unit tests for methods making up the algorithm. So the limitation of TDD is not that it cannot be applied – I am using it – but that it is not enough to ensure what I consider to be a sufficient level of testing. See the Techniques section below for how I address this limitation.
3. In situations where automated unit tests are not applicable, TDD obviously does not apply. Some people would insist that everything be unit tested, and I agree that it is a goal to aspire to, but in some circumstances I feel that unit testing is not pragmatic. Situations where I am unlikely to use automated unit tests and hence TDD include:
   • Prototyping or other exploratory-style work such as an architectural spike. However, when trying to understand the behavior of third-party libraries, I often do find it helpful to do this via unit tests.
   • User interfaces such as web pages or emails. Automated tests can be used to verify that the web page or email content is produced without failure, but the actual content and formatting is best checked by a human.

Techniques

I have adopted several techniques for using TDD to fit my style of coding that go beyond the three rules. I prefer to think of them, however, as tips or tricks of the trade rather than firm rules.

1. I developed a technique to address the limitation of TDD discussed above regarding the creation of new methods on collaborating classes under a single failing unit test. When I go to create a new method on a different class, I recursively apply TDD. So before creating the new method, I create a new test for it on a different test class corresponding to this other class. This means that I now have two failing tests, not one, so I modify the TDD rules and just run the tests of this second class while working on this new method. Once I am done with the method, I can run the suite, confirm I have just the one original failing test, and resume working on the original method.
2. There are often times when, upon getting the current tests to pass, there are multiple scenarios to select from to write the next test. Which one to pick? My own preference is to choose a scenario that will fail given the current production code – do not choose a scenario that will automatically pass. The reason for this preference is that this helps maintain that rhythm of alternating between failing and passing tests. After all of these failure-inducing scenarios are tested, I do go back to add the scenarios I expect to pass. At this point, while I’m still technically following TDD, it feels like my old approach of writing the tests afterwards: the production code is finished, and I am testing the remaining scenarios to confirm it is correct.

Conclusion

Overall I found test driven development to be a very effective process for producing high-quality code, and I plan to continue to use it. I highly recommend every developer to experiment with adopting TDD and evaluate the benefits and limitations for themselves.

Recently I was prompted by multiple sources to give TDD a try, the most prominent and vocal of which was Robert C Martin, who stated that he believed TDD to be one of the most significant new software development practices he had adopted in his 30 years of experience. I found particularly compelling his comparison of TDD for software development to the medical practice of sterile operating rooms.

Based on these strong recommendations, I decided that I needed to give test driven development another try.

Preparation

I resolved to not just haphazardly try TDD like I did before, but to adopt it as a development practice for a period of time as a continuous improvement experiment. I deliberately went with a more disciplined approach. Based on my knowledge of personal development and continuous improvement, I knew that the change would be difficult, especially at first. So I prepared for the change via the following steps:

• I reflected on the difficulties I would face in adopting TDD. I expected to struggle with two issues. The first was the drop in productivity due to getting familiar with doing TDD – I would be spending a greater percentage of my time thinking about my process of coding as it related to TDD, rather than the code I was writing. The second was the natural tendency to revert back to my established pattern of behavior. This reflection ensured I would have more realistic expectations when I started using TDD.
• I refreshed my knowledge of the details of doing TDD that go beyond the core idea of writing tests before code. My favorite reference was Robert C. Martin’s three rules of test driven development which are:
  1. You are not allowed to write any production code unless it is to make a failing unit test pass.
  2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
  3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
• I wrote a note to do TDD and stuck it to the front of my keyboard were it would always be visible to me. It served as both an affirmation and a reminder.

Having finished my preparation, it was time to actually start doing test-driven development.

Adoption

My initial start with TDD was easy: I started my next coding task by writing a test rather than production code. If only it stayed so simple :)

At first the shift in process was difficult as I had to consciously remember to write the test first, and then write only a portion of the production code necessary to get it to pass. My productivity felt a lot lower (I have no idea whether it was significantly worse or only a little). But I had expected this and used discipline to force myself to continue with TDD.

One of the hurdles I faced was how strictly to follow the three rules of TDD - particularly rule number three. I had always been uncomfortable with the idea of writing temporary or intermediate production code that would get the current test to pass, but that I knew was not the final form and that I would need to change. An example is implementing an algorithm to return a fixed value (say zero or null) rather than implement the actual logic. I decided to take a pragmatic approach and allow myself to deviate from the three rules on occasion, when following the rules seemed too onerous or difficult. I did not want blind adherence to the rules to cause me to completely give up on TDD. I expected that over time as my familiarity with TDD grew, I would be able to become stricter in adhering to the rules.

As expected I suffered setbacks along the way. I started a development task by writing most of a method of production code before realizing that I had no failing test. In another case I had a failing test but then churned out the entire production method, including all the special cases, well after that test would pass. I took these setbacks in stride – I considered them a normal outcome of adopting a new behavior, rather than personal failures, and simply returned to doing TDD once I became aware of my departure.

After about a week, doing TDD became less of a struggle. After about three weeks (the typical minimum duration to establish a new habit) TDD began to feel more natural. By this point I had clarified for myself the benefits and limitations of TDD and had integrated it into my development process. I have a lot more to say about this which I will save for a follow-up post. As a quick summary, I find TDD to be a valuable practice that I intend to continue to use.

Conclusion

If you have not tried TDD, I strongly recommend you experiment with adopting it as a development practice. Looking beyond just TDD, one of the points of this article is to encourage you to always be thinking about your capabilities as a software developer and continuously seek to improve.

I use two guiding principles as the basis for constructing my definition.

1. Potentially releasable: Ideally the software can be released (or shipped) once it is done. I've seen many people, particularly in the context of Scrum, use the similar term "potentially shippable".
2. I would trust my life to my code: I just wrote an entire article on this titled Would You Trust Your Life To Your Code?

These principles are deliberately idealistic in order to set high expectations and motivate continuous improvement when I fall short of reaching them.

Different definitions of done can be created based on different levels or scopes. The two primary scopes are:

1. Done for a feature / user story.
2. Done for a release.

For this article I am using my definition of done for developing a feature (user story).

My definition of done is essentially a checklist with items grouped into categories. The lists of items and categories are not meant to dictate the process by which these items are done or the chronological order. For example, automated unit testing is listed in a separate category from coding but it is typically done at the same time or before-hand, if doing test-driven development.

Without further ado, here is my definition of done.

Coding

1. Code meets functional requirements.
2. Code meets non-functional requirements. Typical ones include:
   • Performance (capacity, scalability)
   • Usability
   • Security
   • Maintainability
3. Code is deployment-ready. This means environment-specific settings are extracted from the code base. A past article I wrote on designing for deployability provides more context on this.
4. Code is checked into version control.
5. Code complies with coding & architectural standards.
6. Code has been cleaned up. The goal is to ensure the code is easily readable and has a good design. In the past I have used the terms polishing code and refactoring to describe this. Robert C. Martin's book Clean Code: A Handbook of Agile Software Craftsmanship provides the best explanation of this that I have seen. Achieving this goes a long way towards meeting the maintainability requirement.
7. All TODO-style comments in the code have been addressed and removed.

Static Code Analysis

1. Code has been analyzed by static code analysis tools. The two primary tools I use for Java development are the Eclipse compiler and FindBugs. Other Java tools include PMD, Checkstyle, and Architecture Rules.
2. All errors and warnings found by the tools have either been corrected or have been suppressed with a comment indicating the reason for suppression.

Testing

1. Automated unit tests are written. The tests should be high quality (e.g. not brittle).
2. Automated integration tests are written that verify interactions with external systems such as the application database or third-party application / web services.
3. Code coverage achieved by the automated tests is measured and sufficient coverage is achieved. I use Cobertura for measuring code coverage. I do not like using only a numeric percentage coverage target as the sole definition of sufficient coverage, because this can encourage people to write poor-quality tests that merely execute the code rather than verify its correct behavior in order to meet the target. My true definition of sufficient coverage is that the tests execute and verify all code that could reasonably be incorrect. Having said this, I generally aim for at least 80% line (statement) coverage overall, and often achieve 90%+ coverage for individual classes. I am still debating what a reasonable target is for branch (conditional) coverage . I currently aim for at least 50% overall, but I have the feeling that a target of 75% would be better.
4. Functional testing by someone other than the developer has been done. Ideally this testing will be done by the customer, involve exercising the complete feature being coded in the way that users would use it, and be fully automated. More frequently I have seen this testing done manually (especially for user interfaces) by business analysts or testers who act as proxies for the customer. The key idea is to have someone other than the developer do testing to validate the assumptions and interpretations made (often implicitly) by the developer.

   I use the vague term "functional testing" rather than the more common terms "system testing" or "user acceptance testing" because projects can differ dramatically in what is done for system or user acceptance testing. If acceptance testing is done in a waterfall fashion as a separate phase near the end of the project then it cannot be part of the definition of done for a feature (but it is still part of the definition of done for the release). So I use the term "functional testing" to indicate this potential differentiation. Ideally, based on lean principles, all testing including system and user acceptance testing should be done as part of the work on a feature and not artificially delayed till later.

Reviewed

1. Design / approach has been reviewed by the technical lead / architect.
2. Detailed peer review / inspection has been done. If pair programming is being used then the peer review is automatically done at the same time as the coding. Otherwise, the reviewer should focus on issues that are less likely to be found by the static code analysis or automated testing. This can include items such as security holes, concurrency issues, and correctly meeting requirements.
3. Issues identified by reviewers have been resolved to the reviewer's satisfaction.

Other

1. Required documentation has been updated. This may include online help, user manual, or operations manual.
2. Build and deployment scripts and related configuration files have been updated.
3. No known defects are outstanding unless the customer has agreed to defer them, in which case they should be logged.
4. No known tasks related to the feature are outstanding.

That concludes my definition of done. I would appreciate hearing about what you use for a definition of done. In particular, if there is anything you think should be added or removed from my definition please let me know via a comment below.

public void placeOrder(Customer customer, Order order) {
  if (customer != null && order != null) {
    // ...
  }
}

I am not a big fan of this style of coding, although I do appreciate that it results in more robust code than code that ignores null checking completely. My coding style instead uses precondition checks to assert that parameters or fields are not null before using them. Rewriting the above example in my style results in the following code.

public void placeOrder(Customer customer, Order order) {
  assert customer != null;
  assert order != null;
  // ...
}

These two styles different in their approach to error handling: mine uses the fail fast principle, while the first uses the degrade gracefully principle. See my article on Fail Fast or Degrade Gracefully for details on the differences between the two. Long-time readers may find my preferred style surprising given my stated preference for the degrade gracefully approach. Let me clarify my position. I still do prefer the degrade gracefully approach, especially for the higher level application architecture & design. My use of fail fast at the detailed coding level is based primarily on my philosophical view of coding, which is to use the technical computer language (Java in the above example) to construct a language representing the business problem domain. The placeOrder method becomes part of the syntax of this business language that semantically only makes sense to apply to an actual customer & order. Having the method handle null arguments does not make sense when expressed in terms of the business language. From a more pragmatic viewpoint, having either the customer or order be null almost certainly reflects an error in the calling code. Blindly ignoring null values throughout the code base significantly increases the risk of errors going undetected.

One special case of aggressive null checking that really bothers me is checking for null collections. Below is an example:

public class Customer
  private List orders;
  public List getOrders() {
    return orders;
  }
}
public class CustomerProcessor {
  public void checkOrders(Customer customer) {
    if (customer.getOrders() != null) {
      for (Order order : customer.getOrders()) {
        // ...
      }
    }
  }
}

In terms of the business domain, a customer can have zero or more orders. So the collection can be empty. But the relationship exists for every customer – a null collection has no business meaning. Checking if a collection is null therefore adds meaningless noise to a method. My approach instead is to have a class invariant that the collection is always defined, which then avoids the need for null checking. This is easily accomplished by initializing the collection at the time of definition. Rewriting the above code in my style results in the following:

public class Customer
  private List orders = new ArrayList();
  public List getOrders() {
    return orders;
  }
}
public class CustomerProcessor {
  public void checkOrders(Customer customer) {
    for (Order order : customer.getOrders()) {
      // ...
    }
  }
}

I believe that this style of coding is more readable and understandable, in large part because it more closely matches the problem domain being modeled without the baggage of extra null checks. This approach does require more thought regarding whether fields and parameters can or should be null, which is perhaps one reason why aggressive null checking is often used instead.

My approach can be compared to defining the nullity of columns in a database schema. One advantage with database design is column nullity is an explicit part of the database schema language, whereas programming languages such as Java provide no explicit support for the concept. I have long though that an explicit syntax for nullable versus non-nullable variables would be an interesting experiment. I have seen experimental languages with such a feature but I have not heard of any remotely mainstream language that does so. One possible syntax is to suffix nullable variables with a question mark. The compiler could then verify at compile time that such variables are checked if they are null before being used. Additionally at run time an exception could be thrown if a non-nullable variable is assigned a null value. Below is an example of what such a revision to Java would look like:

public void placeOrder(Customer customer, Order order, Discount? discount) {
  // Customer and Order are not nullable, Discount is nullable
  // Invoking a method on discount here would be flagged as an error by the compiler:
  double percentageDiscount = 1.00;
  if (discount != null) {
    // No error because we have checked if discount is null
    percentageDiscount = discount.calculatePercentage();
  }
  // Can invoke methods on order and customer without checking if they are null.
  order.prepareToBePlaced(percentageDiscount);
  customer.addOrder(order);
}

I am not a language designer so I do not know all the ramifications of such a language change. Therefore I am not saying that this should be added to Java, merely that it is a possibility. If you do want compiler support for dealing with nulls, then one alternative is to use the Eclipse IDE and enable its Java compiler for performing a static compile time analysis to warn of the use of variables that may be null (see image below).

• A higher percentage of defects are found when using reviews – as high as twice as many than testing.
• When an issue is detected by a review, the root cause of the problem is usually known. Defects found by testing require further work to analyze.
• Reviews can be performed earlier in the development life-cycle than testing.
• Reviews provide more precise and more immediate feedback for developers to learn and improve from than testing.

Steve McConnell in his book Rapid Development summarizes these findings on pages 73 to 75.

Given these many benefits, it is disappointing that more teams do not use code reviews. If you are part of a team that does regular code reviews, then I congratulate you. Just because you do code reviews, however, does not guarantee that you will see all the benefits. In fact, I have often observed highly ineffective reviews. One example from my own experience is when a developer was asked to do a review but viewed the task as an annoying interruption from more interesting work. They did a very quick and superficial 'review', and then reported that everything looked fine. As a task the review was completed, but the expected quality improvements were not achieved. In a sense the review did not really happen.

How can you avoid this from happening to you and your team? In this article I present a set of strategies for achieving effective code reviews. These are based on my years of experience performing code reviews and figuring out what works the best. Some of these ideas are inspired by concepts from PhotoReading, a multi-step system for efficiently extracting desired information from written text (aka reading, but not as it is traditionally done).

Adopt a Critical yet Playful Attitude

The primary purpose of a code review is to critique the code - to find defects, potential problems, or simply poorly-written sections – which requires a critical mindset that is different from the typical problem-solving mindset used by developers as they solve code. You cannot simply read over a section of code and accept what you see: you need to challenge the assumptions and decisions behind it. This is especially important for finding errors of omission. It is much harder to find a defect because of what was not written, and to do so requires you have a questioning, suspicious attitude towards the code you are reviewing. You need to mentally prepare yourself before starting a review by reminding yourself of the purpose of the review and the need for a critical, questioning attitude.

Maintaining this critical attitude throughout your review, however, can be difficult, especially if it is not your regular mode of thinking. It helps to cultivate a delight or enjoyment in finding issues with the code, like you are playing a game searching for treasures hidden by an opponent. Unearthing a serious defect should feel equivalent to hitting a home run in baseball. Viewing the review as a fun yet challenging game where you score points for the issues you find will boost your motivation and keep you focused in that critical mindset.

The critical attitude required for reviews is similar to the questioning mindset needed for root cause analysis. In root cause analysis, however, you are trying to determine why something failed, whereas in a code review you are trying to find all potential causes of failure.

Take Multiple Passes Through the Code

In order to do an effective review you must learn how the code is structured, what it is trying to accomplish, and where it falls short. It is difficult to do this all at once as you read through the code for the first time. A much more effective approach is to use multiple passes to review the code with the initial passes focused on learning about how the code works and later passes focused on critiquing the code.

The number of passes you will need will depend on the size of the code base you are reviewing, its complexity, and how thorough a review you are doing. As a starting point, here is a suggested approach for reviewing a code base in multiple passes.

1. Problem Overview: Review the requirements or specification to understand what the code is supposed to do. What problem is it trying to address? What requirements is it implementing?
2. Solution Overview: Review the design document or specification to learn the overall solution to the problem. How are the requirements being addressed?
3. High Level Organization: Review the code or detailed design document to figure out how the code is organized and functions at a high level. What are the packages and the major classes within each? I find UML class diagrams showing class relationships and hierarchies very useful for this pass, and if none already exist as part of the documentation I often end up sketching out such a diagram. UML sequence or state diagrams can also be useful to highlight the main interactions or behavior of the system.
4. Major Class Review: Review the code for the most important classes as identified previously. While issues and problems can be found in earlier passes through the code, particularly at the design level, this pass marks the transition from learning about the code to critically analyzing it for problems.
5. Multiple Detailed Review Passes: The previous passes will have given you a sufficient understanding of the code base to critically review the remainder of the code base. Even at this point, however, it is still valuable to use multiple passes. I explain why in the next section.

Focus on One Goal Per Pass

One common response to the problem of poor-quality reviews is the creation of a review checklist that lists all the issues a reviewer should be looking for. Often these checklists enumerate many specific items and go on for more than a page. Two examples of checklist items are: (1) checking the return codes of system calls, and (2) using prepared SQL instead of SQL literals for parameters. While the idea of checklists has merit, the reviewer is given no guidance as to how to effectively use the checklist. Short-term memory, which can only hold approximately seven pieces of information, is all needed for analyzing the code. Trying to remember a page of issues to look for while reviewing simply does not work.

The solution to this is to focus on a single goal for each detailed review pass through the code. Each goal can be as specific as an individual checklist item, but I find this level of detail is seldom necessary and leads to too many passes. The goals I prefer to use generally correspond to categories of issues. For example, instead of a checklist item such as checking return codes of system calls, I have the goal of reviewing error handling. As I make my review pass with this goal in mind, I can ignore the bulk of the logic and focus instead on how errors might occur and how they are handled.

Unlike the long checklist, this solution is workable because you only need to hold a single piece of information – the current review goal – in your short-term memory while you are reviewing. Focusing on a single goal, however, provides an additional benefit: it actually improves our ability to find problems in the code pertaining to the goal. How does this work? Our eyes and other senses constantly transmit a wealth of information to our brain – too much information, in fact, to fully process in detail. The brain must therefore selectively filter the information it receives. The brain decides what information to select based in part on our goals and desires – what we view as important. In essence, our thoughts and perspective affects how we view the world. When we focus on a single goal such as error handling, as we review the code the brain is more likely to focus on segments of code dealing with error handling (or the lack thereof). This allows you to review the code faster: you can skim along, skipping past sections that do not apply, then slow down to carefully analyze a relevant block or a suspicious section, and then speed back up again. Changing your goal between review passes causes your perspective to shift, allowing you to identify a different set of issues in the same code base. This greatly increases the effectiveness of the review by catching more issues across a wider variety of categories.

Here are some typical goals I use for my reviews:

• Functionality: Does the code meet the business (functional) requirements? Is the logic correct?
• Class Design: Does each class have low coupling and high cohesion? Is the class too large or too complicated?
• Code Style: Is duplication of code avoided? Are any methods too long? Are typical coding idioms / standards followed?
• Naming: Are packages, classes, methods and fields given meaningful and consistent names?
• Error Handling: How are errors dealt with? Does the code check for any error conditions that can occur?
• Security: Does the code require any special permissions to execute outside the norm? Does the code contain any security holes?
• Unit Tests: Are there automated unit tests providing adequate coverage of the code base? Are these unit tests well-written?

Add Review Comments Directly to the Code

How should issues found during the review be reported back to the original developer? One commonly used approach is to create a review document listing all the issues found. For each issue the problematic section of code must be referenced, usually by supplying the file name and line number. Management often prefers the review document because it can be used for process auditing and generating metrics. Exclusive use of a review document for recording issues, however, causes problems for the reviewer and the original developer. For the reviewer, switching from the code to a separate document can easily break their flow, especially since they must record not just the issue, but also the reference back to the code. For the original developer, once they start fixing issues in a particular file, the references (file name and line number) of other issues in the same file can quickly become out of date, especially if refactorings such as move method, extract method, or rename class are performed.

I believe, therefore, that the most effective approach to recording issues is to add them directly to the code base as comments, at the place the issue occurs. Each such comment should be tagged with a special label – I use REVIEW – so that the issues can be easily found later by the original developer. In fact, if you use the Eclipse IDE for Java development, you can define a task tag for this type of comment that will cause all such comments to appear in the Task view. Once the review is completed, the code with the review comments can be checked back into the version control system to be made available to the original developers. If you must produce a review document to comply with your organization's processes, then I would recommend creating the document at the end of the review from the review comments you added to the code.

Summary

Code reviews are a powerful practice for improving software quality. The strategies I have found personally useful in increasing the effectiveness of my reviews are:

• Adopt a critical yet playful attitude.
• Take multiple passes through the code.
• Focus on one goal per pass.
• Add review comments directly to the code.

If you have any additional strategies you have found particularly useful, please let me know.