Adopting Test Driven Development

The boldest idea I heard was in response to my question of how to encourage the adoption of test driven development (TDD). The answer consisted of this process:

1. Clearly communicate to the team the expectation that all production code will be produced with tests.
2. Wait and monitor code check-ins until you see code checked in without tests.
3. Delete this untested code and commit the change.
4. Communicate to the affected developer that you found some code that was not necessary because the tests still passed after removing it. Diabolical laughter optional :)

This sounds extreme, but the team I was talking to actually used this in the formative early stages of building their team and the use of TDD has become a standard practice on the team. I think applying this at the start of a project with a new team is the best timing and makes the most sense, but I may have to try it mid-project (my team, watch out :)

I also learned of a different approach to writing individual tests that basically involves starting at the end and working backwards. Start first with the asserts that specify the desired result, then write the execution of the method under test which covers how the result will be obtained, and then finally write the setup code which provides the necessary inputs. This approach is very much a pull approach (in the lean sense) - each step serves as the specification for what needs to be done in the next step. As such, it might be an easier approach for newcomers to unit testing.

Adopting Pair Programming

I asked about dealing with resistance to adopting pair programming, especially the tendency of pairs to collaborate jointly for only a short time before reverting to working separately, each at their own computer. The answer was to take away someones computer, which leaves them with no choice but to pair with someone else on the team.

Environment factors can also help or hinder pairing. The Calgary teams have their workstations set up to allow people to sit side-by-side in front of the same machine. I think this layout is very important for successful long-duration pairing, and I switched my own cubicle over to this layout a while ago. The Calgary teams, however, took this setup one step further and have a second keyboard and mouse for each of their workstations. This has several benefits in terms of encouraging pairing:

• It communicates visually the expectation that two people will work at this computer rather than one.
• It eliminates concerns over using another person's keyboard. The 'owner' of that workstation has their dedicated keyboard, with the second keyboard reserved for guests. Whether the concern is over germs or over intruding on someones property/space, having only one keyboard produces a reluctance on the part of the visitor to drive.
• A common problem in pairing is for the person who is not driving to become too passive. Providing a second keyboard gives them the opportunity to jump in at any point (even when the driver does not want to relinquish control), and having the keyboard waiting in front of them is a subtle reminder to remain actively involved.

Using Task Boards

All the teams use task boards consisting of cards in various columns on a whiteboard, wall, or even a window. My team has recently switched to using a task board and I really like it in terms of it visualizing the work in progress and serving as a hub for discussions, like in the daily scrum. Given my experience and its broad use in Calgary, I am going to more actively promote its use.

Every team seems to have variations in how the task boards are used, but there were some commonalities across the Calgary teams. All teams used large cards - about the size of half a normal 8.5 x 11 page. Compared to the small post-it-note-sized cards my team uses this is a lot more real estate. These cards are made from colored paper cut in half that allows different colors to be used for different types of activities (e.g. defects, stories, and tasks). One team printed out defect reports (from JIRA) on regular paper and folded them in half to attach them to the wall. Another team attached printed requirements (with tests) folded in half to user stories. The teams all had relatively few columns for their task board - usually not more than four consisting of "In Sprint", "In Progress", "In Testing", and "Done".

Professional Development

The Calgary teams have a very strong culture of professional development that manifests in a number of ways. The most visible manifestation is my seeing many recent software development or I.T. books scattered around the office, many of which are books on my reading list (I may have to do an office raid :).

What I consider the most significant manifestation of this culture is the existence of a professional development group for senior developers. This group holds a number of activities that include periodic presentations by members on technical topics, book reading groups to jointly go through particular books, and Friday noon viewings of videos relating to software development.

One of the technical leads had an interesting rationale for doing professional development: not doing it is essentially stealing from your clients in the future. If you spend five percent of your time doing professional development and improve by five percent per year, then you will break even after the first year and show a benefit in future years: your output will be higher than before, even after subtracting the five percent. So not doing professional development will leave your client shortchanged in the future. If you let that trend continue long enough, you will eventually lose your client.

Summary

I had a great time and enjoyed being challenged with new ideas and perspectives on software development. I believe that visiting other teams or companies and seeing first-hand how they do things is quite beneficial. I encourage you to look for opportunities to do this and I hope to do more of it in the future. Thanks CGI Calgary!

In the past my general pattern when doing code reviews has generally been to focus on production code and do a more minimalistic review of the unit test code. But over the past year, I have been seeing indications that this might be a mistake. One respected colleague even advised me to reverse my priorities and focus primarily on the tests. So why would reviewing test code be so important?

Reasons to review test code

Research by Caper Jones indicates that errors in test cases are quite common - as high as 20%. In other words, one in five test cases has something wrong. Caper was likely referring to manual test scripts rather than automated test code, but there's no reason to expect the act of automation to magically fix all such errors. I recently reviewed a set of unit tests for a class and found one test that made no sense - it appeared to be verifying behavior against the wrong expected result. When I raised the issue with the developer, they confirmed that the test was wrong. Fixing the test revealed a defect in the production code which they then promptly fixed. This was the same production code that I had already carefully reviewed.

Automated tests can have other issues besides simply being incorrect. Tests can be incomplete, weak, fragile, or hard to read and maintain.

Incomplete tests fail to test certain functionality. Reviewing the results of a code coverage tool is a very easy way to identify gaps in coverage of the existing production code. A more careful review of the actual tests is needed, however, to ensure that all required functionality is actually implemented since code coverage cannot identify gaps when production code is missing.

Weak tests have a low likelihood of detecting defects. One example is tests for trivial getters and setters - such tests are so weak that I generally consider them useless clutter and delete them when I see them in a review. Test weakness is really a continuum from virtually useless, as per my example above, to extremely powerful. Part of reviewing tests is to identify when the power of the test can be improved. A while ago I had to change a business rule to use less-than-or-equals logic rather than less-than logic as originally specified. I confirmed there were tests covering this rule, then changed the logic and re-ran the tests expecting them to fail, but to my surprise they all passed. Upon investigating, I discovered that the tests improperly exercised the boundary condition - they tested a value below the boundary and above the boundary, but not on the boundary. So the equal-to-boundary case, which was the logic I was changing, was not tested. This prompted me to review the tests for other similar functionality, where I found the same pattern of weak boundary tests, a few of which had led to defects escaping detection. If we had been more diligent reviewing the unit tests when this functionality was first written we could have avoided these problems.

Fragile tests have a high likelihood of requiring updates when production code is changed. In other words, changes are likely to cause such tests to start failing. Having too many fragile tests can lead to maintenance issues down the road - developers may choose to delete such failing fragile tests rather than fixing them, or worse might choose to stop running the tests altogether. Martin Fowler termed this test cancer. Reviews by individuals experienced in maintaining unit tests are the only way I know of detecting and minimizing fragile tests.

Test code needs to be clean - easy to read and maintain - just like production code. In fact, many proponents of automated tests argue that well-written tests can function as documentation highlighting what the production code is supposed to do and providing examples of how to use specific APIs. In my experience, using tests as documentation only works when the test code is clean, which generally requires reviews to ensure it happens consistently.

Incorrect perceptions regarding reviewing test code

Given all these compelling arguments for reviewing unit tests, why have I not reviewed tests as much in the past? The 'excuses' I have used seem to be due to incorrect perceptions on my part.

As an architect, I often do what I call architect-level reviews, focusing on the overall structure of the production code and non-functional attributes. I am often not familiar with the detailed functional requirements the code is trying to satisfy. So I don't bother looking at the details of the test code, since I usually can't confirm if the expected results are actually what the requirements specify. But wait - if I am able to still effectively review the production code, then I can perform the same type of review for the test code.

I focus first on production code due to my underlying belief that correct and clean production code is more important than test code. In one sense, this is true. But there are other viewpoints. One view is that if it isn't tested, it doesn't work. Another view is that the long-term vitality of the production code over multiple release depends on a solid, well-written suite of automated tests.

Related to the prior error in perception is the issue of schedule and/or budget constraints. Given limited time or effort, where should I allocate my time for a review? If my priority is production code, then I focus on that and neglect the test code. But this very question contains a flawed assumption: it assumes I should sacrifice the thoroughness of the review for other constraints. But the defects I fail to detect as a result of this sacrifice will likely get caught and fixed later at much greater expense. Reviews are one of the most effective means of defect removal - they are usually twice as effective as most kinds of testing at finding defects and are half as costly to fix the defects that are found.

Conclusion

While I hope this article helps convince you of the importance of reviewing test code, I wrote it just as much for myself to reinforce the importance of doing these reviews. Here are some pointers for getting started:

• Use a code coverage tool like Cobertura to review how well the tests cover the production code.
• Review the tests first, before reviewing the production code.
• Use the following questions as a guide to reviewing test code:
  • Are the requirements fully tested? Are there any omissions of functionality?
  • Are the tests weak or fragile?
  • Are the tests readable and the code clean (e.g. no duplication)?

This rule is based on a bottom-up engineering philosophy: if you completely understand the methods you are invoking, then you should understand the behavior of the method you are coding and know that it will work. This applies recursively up the call stack to the top-level entry point of your application.

Key Requirements

My formulation of the rule above specifies two key requirements for using another method:

1. Behavior Understood: The behavior of a method can be defined in terms of its pre-conditions and post-conditions. Knowing the pre-conditions allows you to ensure you are correctly invoking the method, while knowing the post-conditions ensures that you will get the results you want.
2. Confident Will Work: This is arguably part of the prior requirement - understanding a method’s actual (rather than stated or expected) post-conditions - but it is so important I felt it should be explicitly stated. You need to verify that the method you are using will actually function correctly and not fail due to a defect. This verification is best done via automated tests, but there are scenarios I discuss below when another approach is needed.

Applying the Rule

Applying the Use Understood Methods Rule involves, therefore, satisfying these two requirements. Exactly how I do this varies based on the context – specifically the nature of the method I intend to use. Below I describe the common scenarios I encounter and the approach I use to apply the rule to each.

• Method with implementation in code base: Calling a method that has an existing implementation within the code base you are working on is the most common scenario. This can be a method on the same class, on a super class, on a collaborating class, or a static method. This also can be an abstract method defined on an interface or abstract base class for which the implementing class exists and is known. To understand the method’s behavior I refer to the documented pre- and post- conditions if available, otherwise I look at the source code for the method. The correctness of the method should already be verified through automated tests. If necessary I can use code coverage analysis results to confirm that this method has sufficient test coverage.

• Method to be written concurrently in code base: Often when coding a method, I find I have to create a new method, either on the same class or on a separate class. This might be a simple extract-method refactoring of logic to simplify the existing method, or it might be new functionality required on another class. In this scenario I have no issues understanding the behavior of the new method as I am writing it at the same time. If the new method is non-trivial then I ensure it works by creating unit tests for it separate from my tests for the original method I am working on.

• Method with unknown implementation in code base: This applies to abstract methods in interfaces and abstract base classes for which no implementation yet exists or for which the implementation cannot be known in the context of the method being written. This latter scenario is typical when there are multiple implementations being processed in a common fashion. In this case I insist on having documented pre- and post- conditions for the method being invoked.

  If the method implementation does not yet exist then it is obviously not possible to verify now that it is correct, but it is possible to take steps to gain confidence that the implementation will work once it is written. One option is to ensure that the automated tests for the method you are writing that invokes this not-yet-implemented method sufficiently exercises the functionality of this not-yet-implemented method to ensure it will meet your needs. Another more general option is to ensure that the team has processes in place such as test driven development and code reviews to ensure that code written in the future will indeed work.

• Method in third-party code: When using third-party methods I usually rely on the documented API. When this is inadequate I look at the source code if it is available (this is a key benefit of open source libraries – the code is always available). In some cases the third-party software implements a specification (such as the various Java EE specs) and the specification can be used to understand what the third-party code is supposed to do.

  Once third-party software has been selected for use within a project I generally assume it works. The verification of the quality of such software happens previously, in the selection process, when I do my due diligence to evaluate the quality of the software under consideration.

  On rare occasions I write a unit test to understand and/or verify how some third-party functionality works. This is something I would probably benefit from doing more often.

Given the prevalent use of automated tests to verify correct behavior, you might be wondering how the application of this rule is impacted by the use of test driven development (TDD). The short answer is there is no impact: this rule applies the same whether or not TDD is being used. I do, however, slightly revise how I do TDD in order to apply this rule for the scenario Method to be written concurrently in code base - I create a second failing test before creating the new method. For further details see my article describing this refinement to TDD.

The Broader Context

Following the Use Understood Methods Rule is necessary for creating high quality code that you would trust your life to, but is not sufficient. Correctness also depends on satisfying class and application-wide invariants (such as properly closing database connections or limiting the number of file handles consumed concurrently), understanding the behavior of containers and frameworks (such as the Spring application context or Java EE container), and considering other quality attributes (such as security and scalability) which are and emergent in nature and not easily analyzed by looking at methods independently. I consider my rule to be the foundation on which the code is written, which these more global concepts rest on top of.

Benefits

This section covers the benefits, as I see them, of doing TDD. This does not include the benefits of doing automated unit testing, which I am a big fan of and have been doing for years using a non-TDD approach (i.e. writing tests after writing production code).

1. Using TDD provides great code coverage, especially conditional code coverage. Strictly following Robert C. Martin’s three rules of TDD should result in 100% coverage for both statements and conditionals. I allow myself to deviate from these rules at times, but still obtain 90+% coverage.

   Previously when writing tests after coding, as part of my process to ensure I was done a feature I would check the code coverage results to identify gaps and add missing tests. I have found that this code coverage check is mostly a formality when doing TDD.

2. TDD helps avoids the tedium that I have experienced at times writing the tests after coding. Often while doing TDD I am able to establish what I call the red-green rhythm: write a failing test (unit test result bar goes red), then get it to pass (bar changes to green). This rhythm makes it more enjoyable to write the tests, although the tedium is not completely eliminated. As a result, I find it takes less discipline to write the tests first than afterwards – I do not have to force myself to write a bunch of tests after getting some functionality in place.
3. When I started using TDD I was initially uncomfortable with writing the simplest possible code to get a failing test to pass (TDD rule #3) when I knew that the final production code would be different. As my experience using TDD increased, I began to see a number of benefits of following this rule:
   • Writing more code than the minimum necessary to pass the test runs the risk of having logic in the production code (statements or conditions) not covered by the tests.
   • At times, when the design for the method / class was a little fuzzy to me, writing the simplest possible code actually proved helpful, as I could then write another failing test which then clarified for me what the design would need to be.
4. Using TDD, especially when strictly following the three rules, seems to eliminate the question / debate about how much to test. When introducing unit testing to developers (not TDD, just the use of automated unit tests), I get asked this question time and time again. I have a standard answer I use, but it no longer seems relevant when doing strict TDD. In fact, the question itself no longer applies when doing TDD.

Limitations

When adopting a new practice it is important to know the contexts in which the practice is less applicable. I have come across a number of situations in which TDD seemed less helpful. These were the situations when I was most likely to deviate from the three rules of TDD or abandon TDD entirely.

1. I prefer to have the design of the method / class I am working on fairly clear in my head before I start writing tests. Sometimes I do this on paper, but sometimes I do this by working with the actual code which would be a deviation from TDD. Since adopting TDD I have tried to do this design in the test code instead of in production code, with mixed results. Sometimes this worked fine, and sometimes it felt unnatural and less productive. I have the feeling that as I gain experience with TDD I will grow more comfortable with doing design within test code.
2. When I need to code a non-trivial algorithm I often extract logic into separate methods on the same class or need to invoke new methods on collaborating classes. Often I need to write these additional methods as the minimum needed to get my failing unit test to pass, which means that I am strictly following TDD. The issue is that my normal unit testing practice is to test methods individually as much as possible, especially methods on other classes, and the rules of TDD do not require me to do so. In essence, my original failing unit test ends up being an integration-style test for the overall algorithm, while I want to have individual unit tests for methods making up the algorithm. So the limitation of TDD is not that it cannot be applied – I am using it – but that it is not enough to ensure what I consider to be a sufficient level of testing. See the Techniques section below for how I address this limitation.
3. In situations where automated unit tests are not applicable, TDD obviously does not apply. Some people would insist that everything be unit tested, and I agree that it is a goal to aspire to, but in some circumstances I feel that unit testing is not pragmatic. Situations where I am unlikely to use automated unit tests and hence TDD include:
   • Prototyping or other exploratory-style work such as an architectural spike. However, when trying to understand the behavior of third-party libraries, I often do find it helpful to do this via unit tests.
   • User interfaces such as web pages or emails. Automated tests can be used to verify that the web page or email content is produced without failure, but the actual content and formatting is best checked by a human.

Techniques

I have adopted several techniques for using TDD to fit my style of coding that go beyond the three rules. I prefer to think of them, however, as tips or tricks of the trade rather than firm rules.

1. I developed a technique to address the limitation of TDD discussed above regarding the creation of new methods on collaborating classes under a single failing unit test. When I go to create a new method on a different class, I recursively apply TDD. So before creating the new method, I create a new test for it on a different test class corresponding to this other class. This means that I now have two failing tests, not one, so I modify the TDD rules and just run the tests of this second class while working on this new method. Once I am done with the method, I can run the suite, confirm I have just the one original failing test, and resume working on the original method.
2. There are often times when, upon getting the current tests to pass, there are multiple scenarios to select from to write the next test. Which one to pick? My own preference is to choose a scenario that will fail given the current production code – do not choose a scenario that will automatically pass. The reason for this preference is that this helps maintain that rhythm of alternating between failing and passing tests. After all of these failure-inducing scenarios are tested, I do go back to add the scenarios I expect to pass. At this point, while I’m still technically following TDD, it feels like my old approach of writing the tests afterwards: the production code is finished, and I am testing the remaining scenarios to confirm it is correct.

Conclusion

Overall I found test driven development to be a very effective process for producing high-quality code, and I plan to continue to use it. I highly recommend every developer to experiment with adopting TDD and evaluate the benefits and limitations for themselves.

Recently I was prompted by multiple sources to give TDD a try, the most prominent and vocal of which was Robert C Martin, who stated that he believed TDD to be one of the most significant new software development practices he had adopted in his 30 years of experience. I found particularly compelling his comparison of TDD for software development to the medical practice of sterile operating rooms.

Based on these strong recommendations, I decided that I needed to give test driven development another try.

Preparation

I resolved to not just haphazardly try TDD like I did before, but to adopt it as a development practice for a period of time as a continuous improvement experiment. I deliberately went with a more disciplined approach. Based on my knowledge of personal development and continuous improvement, I knew that the change would be difficult, especially at first. So I prepared for the change via the following steps:

• I reflected on the difficulties I would face in adopting TDD. I expected to struggle with two issues. The first was the drop in productivity due to getting familiar with doing TDD – I would be spending a greater percentage of my time thinking about my process of coding as it related to TDD, rather than the code I was writing. The second was the natural tendency to revert back to my established pattern of behavior. This reflection ensured I would have more realistic expectations when I started using TDD.
• I refreshed my knowledge of the details of doing TDD that go beyond the core idea of writing tests before code. My favorite reference was Robert C. Martin’s three rules of test driven development which are:
  1. You are not allowed to write any production code unless it is to make a failing unit test pass.
  2. You are not allowed to write any more of a unit test than is sufficient to fail; and compilation failures are failures.
  3. You are not allowed to write any more production code than is sufficient to pass the one failing unit test.
• I wrote a note to do TDD and stuck it to the front of my keyboard were it would always be visible to me. It served as both an affirmation and a reminder.

Having finished my preparation, it was time to actually start doing test-driven development.

Adoption

My initial start with TDD was easy: I started my next coding task by writing a test rather than production code. If only it stayed so simple :)

At first the shift in process was difficult as I had to consciously remember to write the test first, and then write only a portion of the production code necessary to get it to pass. My productivity felt a lot lower (I have no idea whether it was significantly worse or only a little). But I had expected this and used discipline to force myself to continue with TDD.

One of the hurdles I faced was how strictly to follow the three rules of TDD - particularly rule number three. I had always been uncomfortable with the idea of writing temporary or intermediate production code that would get the current test to pass, but that I knew was not the final form and that I would need to change. An example is implementing an algorithm to return a fixed value (say zero or null) rather than implement the actual logic. I decided to take a pragmatic approach and allow myself to deviate from the three rules on occasion, when following the rules seemed too onerous or difficult. I did not want blind adherence to the rules to cause me to completely give up on TDD. I expected that over time as my familiarity with TDD grew, I would be able to become stricter in adhering to the rules.

As expected I suffered setbacks along the way. I started a development task by writing most of a method of production code before realizing that I had no failing test. In another case I had a failing test but then churned out the entire production method, including all the special cases, well after that test would pass. I took these setbacks in stride – I considered them a normal outcome of adopting a new behavior, rather than personal failures, and simply returned to doing TDD once I became aware of my departure.

After about a week, doing TDD became less of a struggle. After about three weeks (the typical minimum duration to establish a new habit) TDD began to feel more natural. By this point I had clarified for myself the benefits and limitations of TDD and had integrated it into my development process. I have a lot more to say about this which I will save for a follow-up post. As a quick summary, I find TDD to be a valuable practice that I intend to continue to use.

Conclusion

If you have not tried TDD, I strongly recommend you experiment with adopting it as a development practice. Looking beyond just TDD, one of the points of this article is to encourage you to always be thinking about your capabilities as a software developer and continuously seek to improve.

The major change in this release is added support for aggregated reporting across different task categories, which I call rollup groups. This required expanding the categories of types to three: base tasks, rollup tasks, and aliases. This led to significant changes to the format of the task definitions file. These changes are backwards compatible: a version 1.X task definitions file will be interpreted as mapping aliases to base tasks. The report format was changed slightly - times for aliases are not reported, and the period report section is revised with the inclusion of the aggregated times for the rollup tasks in each rollup group. For more details see the updated user manual supplied as part of the release.

Software Development Lessons Learned

Once again I was reminded of the importance of careful testing. Despite a thorough suite of unit tests, I was embarrassed to discover that my manual execution of the example files provided in the release failed due to a defect. This brought to my awareness several gaps in my testing: I was not automatically testing the example files, I had no realistic end-to-end automated system test, and my unit tests were obviously missing some scenarios. I spent some time correcting these gaps.

Believing I was now finished, I prepared my first beta release and put it to use. I quickly discovered deficiencies and even a defect as I iterated through a few beta releases to get to this final 2.0 release. I found it interesting that this release pattern matches what I described in my article Cyclic Incremental Growth in Software Systems (assuming that the beta releases are counted as actual releases). Hopefully I won't discover the need to make a 2.0.1 release.

Update February 21, 2009: I just released version 2.0.1 to fix some defects in error handling when trying to parse invalid dates. Lessons learned: I need to remember to test realistic invalid input for parsers and I need to be more paranoid about potentially null references.

Tutorial Introduction Previous: Initial Setup

Determine What to Test

Before you can write your first unit test you need some code to test. If you are trying this in your own project, then select a class with a method that needs unit testing. How do you know what code needs testing? The general criteria is this: when you look at the code for the class, if there is any logic which you are uncertain will work in all scenarios, then you should write unit tests for that logic. To be specific, one-line methods like trivial getter and setter methods do not need testing – it is obvious that they will work. Most methods longer than a few lines of code and especially those with conditionals or loops usually warrant unit tests.

You must be wary of false confidence in your code. You need to be one hundred percent confident that the code works as intended across all scenarios – including bad input, system failure, null references, etc. - in order to decide it does not need a unit test.

As you are choosing your class to test, it will be helpful for this first time to choose a class with minimal dependencies: the class should require a minimum (ideally none) of other classes / objects to be supplied for the class to be used.

In the remainder of this section I will refer to the class and method you have chosen as class YourClass located in package your.package and method yourMethod().

To provide a concrete example I will be testing the class ParserHelper from the example project UnitTestingTutorial. The code for this class is shown below:

// Copyright 2009 by Basil Vandegriend.  All rights reserved.
package com.basilv.unittestingtutorial.writeyourfirsttest;

/**
 * Helps parse a set of tokens. Maintains an internal index into the
 * supplied tokens to determine which token all the methods will act on.
 * This index must be advanced by the caller.
 */
public class ParserHelper
{
  static final String DEFINITION_SEPARATOR = "->";

  private String[] tokens;

  private int currentTokenIndex;

  public ParserHelper(String[] tokens) {
    this.tokens = tokens;
  }

  public void nextToken() {
    currentTokenIndex++;
  }

  public String extractTaskName() {
    if (!haveMoreTokens()) {
      throw createParseException("Expected task name but no further input supplied.");
    }
    return currentToken();
  }

  public String extractRollupGroup() {
    if (!haveRollupGroup()) {
      throw createParseException("Expected a rollup group of the form '['']'.");
    }
    String token = currentToken();
    String group = token.substring(1, token.length() - 1);
    if (group.length() == 0) {
      throw createParseException("An empty rollup group '[]' was specified.");
    }
    return group;
  }

  public boolean haveRollupGroup() {
    if (!haveMoreTokens()) {
      return false;
    }
    String token = currentToken();

    return (token.startsWith("[") && token.endsWith("]"));
  }

  public void expectSeparator() {
    if (!haveSeparator()) {
      throw createParseException("Expected separator '"
        + DEFINITION_SEPARATOR + "'.");
    }
  }

  private boolean haveSeparator() {
    if (!haveMoreTokens()) {
      return false;
    }
    return DEFINITION_SEPARATOR.equals(currentToken());
  }

  private boolean haveMoreTokens() {
    return currentTokenIndex < tokens.length;
  }

  private String currentToken() {
    return tokens[currentTokenIndex];
  }

  private RuntimeException createParseException(
    String parseErrorMessage) {
    int tokenNumber = currentTokenIndex + 1;
    String message = "Error parsing token #" + tokenNumber
      + ":" + parseErrorMessage;

    return new RuntimeException(message);
  }
}

This class has a number of methods. Which one should I test? The nextToken method is trivially correct so I skip it. The extractTaskName method is almost simple enough to skip, but I resist the temptation and choose it. (Because the method is simple it will be easy and quick to test.)

Create the Unit Test Skeleton

Before you can write the actual unit test code to invoke the method you chose and verify the results, you must create the skeleton code that JUnit needs to execute the unit test. This skeleton test code follows a standard pattern which is explained in the following steps:

1. Create a test class for class YourClass. The test class will be located in the same package your.package but under the source directory "src/test" instead of "src/main". Name the class YourClassTest. If the class you are planning to test is located on the file system at "src/main/your/package/YourClass.java" you will now have a test class located at "src/test/your/package/YourClassTest.java".
2. Populate the test class using the following template. Include the import statements. You will need to change the name of the class in the template to YourClassTest.

   import static org.junit.Assert.*;
   import org.junit.*;
   
   public class TemplateClassTest
   {
     @Before
     public void setUp() throws Exception {
   
     }
   
     @After
     public void tearDown() throws Exception {
   
     }
   
     @Test
     public void templateMethod() throws Exception {
       fail("TODO: Write this test.");
     }
   
   }
   

   This template provides a single test method identified by the @Test annotation. JUnit will treat this method as a test and execute it as part of the project's suite of tests.

   The @Before and @After annotations indicate methods that are to be invoked before and after the test method. Collectively these two methods represent what is called a test fixture, and are used to create a common set of objects shared across multiple tests.

3. Rename method templateMethod() to yourMethod(). The test code you will write will go in this method.
4. Run the unit tests for the project to confirm the test skeleton was created properly. The Initial Setup section of the tutorial explains how to do this in detail, but as a quick reminder open the menu for the Run icon in the tool bar and choose the JUnit entry named after your project. The tests should now run (without the warning message about having no tests that you encountered during initial setup).
5. The results of running JUnit tests are displayed within an Eclipse view titled "JUnit". See the image below. (Your JUnit view may be laid out differently depending on your screen and options, but will contain the same information.) The most important element of the view is the horizontal red bar near the top, which is a striking visual indicator of the overall status of the tests: red indicates that at least one test was unsuccessful, whereas green is displayed only if all the tests are successful. Above the bar are the summary statistics that indicate the number of tests that ran (1 / 1), the number that encountered an error (0), and the number that failed to meet the expected results (1). Below the red bar in the first pane is the hierarchy of tests, usually filtered to only show the unsuccessful tests. The second pane shows the stack trace of the point of failure or error for the test selected in the first pane.
   
   Your single test is failing because the test method provided in the template includes the following code:

   fail("TODO: Write this test.");

   The fail() method is a static method from the JUnit Assert class that causes the test to fail if it is executed. Including this code in the template provides an automatic reminder to actually write the test as it starts out failing.

Returning to the example project, since I am planning to test the class ParserHelper I need to create a test class ParserHelperTest. The skeleton code for this test class is located below:

// Copyright 2009 by Basil Vandegriend.  All rights reserved.
package com.basilv.unittestingtutorial.writeyourfirsttest;

import static org.junit.Assert.*;
import org.junit.*;

public class ParserHelperTest
{
  @Before
  public void setUp() throws Exception {

  }

  @After
  public void tearDown() throws Exception {

  }

  @Test
  public void extractTaskName() throws Exception {
    fail("TODO: Write this test.");
  }
}

Because I plan to test the method extractTaskName on ParserHelper, I have given the test method the same name.

Write the Test Code

You are now ready to write the actual test code that will exercise the class you are testing and verify the results. While the specifics will of course vary from class to class and method to method, there are three common phases that must be performed in each case: setup, execution, and verification.

1. Setup: Create the object to be tested and configure its initial state prior to being tested. This includes configuring any required dependencies, which includes parameters to the method being tested.
2. Execution: Invoke the method being tested and capture the return result.
3. Verification: Confirm that the method performed the expected actions. This is done by invoking appropriate methods of the JUnit Assert class such as the fail() method introduced above. If an expectation fails then the JUnit method will throw an special exception indicating the test has failed. Execution of the test method will stop and JUnit will record a failure for the test. If at any point in the test method any other exception is unexpectedly thrown (i.e. not caught within the test method) JUnit will record an error for the test.

To demonstrate how to implement these steps I continue the example from above to test the ParserHelper.extractTaskName() method.

1. Setup: I need to create an instance of ParserHelper, which requires an array of tokens to be supplied to its constructor. I chose to supply a single token, which will be the task name that I expect to have returned from the method.

       String expectedTaskName = "test";
       ParserHelper helper = new ParserHelper(new String[] { expectedTaskName });
   

2. Execution: I invoke the method under test and record the actual result.

    String actualTaskName = helper.extractTaskName();
   

3. Verification: I use the JUnit assertEquals method to verify that the actual task name returned matches what I expect.

       assertEquals(expectedTaskName, actualTaskName);
   

The completed test method is shown below:

  @Test
  public void extractTaskName() throws Exception {
    // Setup
    String expectedTaskName = "test";
    ParserHelper helper = new ParserHelper(new String[] { expectedTaskName });

    // Execution
    String actualTaskName = helper.extractTaskName();

    // Verification
    assertEquals(expectedTaskName, actualTaskName);
  }

This concludes the tutorial. There is, however, still much more to learn about unit testing. Check here regularly or subscribe to this site's feed to find out when new sections are added to this tutorial. In the meantime, here are some links to additional resources on unit testing:

• JUnit Cookbook
• JUnit Frequently Asked Questions
• My article on how to write good unit tests by avoiding common pitfalls

Tutorial Introduction Previous: Initial Setup

Previous: Tutorial Introduction Next: Writing Your First Unit Test

Install JUnit

Follow these steps to install JUnit:

1. Download Junit. You just need the individual jar file, but you may want to grab the zip file that also includes the source code and documentation..
2. Place the junit jar file on the classpath for your project:
   1. Create a directory called "libs" under your project root directory.
   2. Copy the file "junit-4.5.jar" you downloaded into this directory.
   3. Ensure that the directory and jar file are visible within Eclipse. If not, right-click on the project and select "Refresh" to synchronize Eclipse with the underlying file system.
   4. Within Eclipse navigate to the junit jar file, right-click on it, and choose "Build Path | Add to Build Path".

Configure Eclipse

Follow these steps to configure Eclipse to run your project's tests:

1. Within Eclipse right-click on the project and choose "Run As | JUnit Test". A dialog should appear with the message "No tests found with test runner 'JUnit 4.0'.". This message appears because we have not yet created any unit tests. You will fix that in the next section of this tutorial. Despite the message, performing this action created the run configuration.
2. Right-click again on the project and choose "Run As | Run Configurations...". This will display the "Run Configurations" dialog box that allows you to edit and create run configurations. On the left side is a tree listing all the run configurations. Under the node "JUnit", click on the run configuration with a name corresponding to the name of your project. (This will be "UnitTestingTutorial for our example.) Note that under the tab "Test" on the right it indicates that this configuration will "Run all tests in the selected project".
3. Select the tab "Common", and within the section "Save As" choose the option "Shared file". This creates a run configuration file within your project which you can check into version control and thus make available to the rest of your team. By default this launch file will be saved to your project's root directory. Within the section "Display in favorites menu" select the option "Run". Click the button "Apply" to save your changes, then select "Close".
4. Check your project's root directory to confirm that the run configuration file was created. It will be named the same as your project name in Eclipse with an extension of ".launch".
5. Within Eclipse expand the favorites menu of the Run button in the Toolbar as per the image below. The top entry of this menu should correspond to the run configuration you just made - it should be named the same as your project with a JUnit icon. Now ignore the menu and click the Run button. This will run the top entry of the menu, executing the project's unit tests. The message that no tests were found should appear again.

Configure Your Project

Follow these steps to configure your project for the unit test code you will write:

1. Create the directory structure "src/test" under the root directory of your project. This is where the unit test code you will write will go. In Eclipse right-click on this "test" directory and select "Build Path | Use as Source Folder".
2. Create the directory structure "src/main" under the root directory of your project. This is where the actual project code should go. In Eclipse right-click on this "main" directory and select "Build Path | Use as Source Folder". If your existing project has the source code under a different directory, move it under "src/main", right click on the other directory and choose "Build Path | Remove from Build Path".

This concludes the initial setup. You can now execute tests for your project, but without any tests this is not too interesting. Fortunately you are now ready to create your first unit test and this is the topic of the next section of this tutorial.

Previous: Tutorial Introduction Next: Writing Your First Unit Test

What are automated unit tests? They are a type of testing performed by developers to ensure that the code they write works as intended. They are automated: tests are expressed as test code that automatically invokes the code to be tested and verifies that the results of execution match the developer's expectations. The scope of testing is individual units of functionality within the code base, such as a single class or method.

This tutorial is oriented towards beginners and therefore is prescriptive. It provides a single step-by-step method for setting up and writing unit tests, usually without explaining the reasons behind why a particular method was used or what some alternative options may be.

This tutorial assumes you already have a Java project with some code (maybe only a single class) set up in the Eclipse IDE. As you go through the tutorial, you should apply the instructions to your project. The tutorial uses a sample project called UnitTestingTutorial to provide concrete examples when appropriate. Use the link below to download this project.

Download the Unit Testing Tutorial Example Project v1.0

This tutorial is divided into a number of sections, each of which is presented as a separate page:

• Section 1: Initial Setup: How to install JUnit and configure your Java project to use it.
• Section 2: Write Your First Unit Test: Create your first automated unit test by writing test code using JUnit and watch the test pass.

I plan to add additional sections to this tutorial over time. If there is particular content you are interested in let me know in the comments below.

Next: Section 1: Initial Setup

I believe that producing good software is hard, and that we software developers need all the help we can get in developing software. I have put together a list of the top five practices that I find to be the most helpful in making it easier to produce good software:

• Automated Tests
• Version Control
• Refactoring
• Repeatable Build and Deploy
• Communication

These practices are generally applicable across the board: for new application development and ongoing application maintenance, for small teams and large teams, for custom enterprise software for a single customer and commercial shrink-wrap software, and for any language or technology. The specifics of how the practice is implemented will of course vary depending on the nature of the project.

Automated Tests

Automated unit tests provide a convenient way to test the software's functionality both during initial development and later as the software is maintained. This is especially essential during maintenance when the original developers are no longer around. Another benefit of automated tests is that they impose a design constraint upon the software that it be testable. This frequently helps improve the overall design of the software by reducing the amount of coupling between different components.

In order to be effective at finding defects, automated tests need to be written effectively. I have come across many examples of poorly written unit tests that are ineffective at finding defects. My article How to write good unit tests discusses this in detail.

Version Control

The use of version control software to manage changes to the software is indispensable for teams larger than one person. Some of the more important benefits of version control include helping coordinate concurrent changes to the code base, providing a history of changes, tracking the versions making up a release, and supporting parallel development on different versions of the software. I even use version control for my personal projects at home, where I am the sole developer. (Read about my experience using Subversion.)

Some of the problems I have observed when version control is not used are: each team member developing with a slightly different version of the software, resulting in chaos when it is time to deploy the software, and variations in the software installed in different environments, with no one able to explain for sure what the correct version is.

Refactoring

The design of a piece of software is a critical attribute for determining its quality. While good design does not imply good software, bad design generally does imply bad software. Refactoring is the key practice by which the quality of the design can be maintained and even incrementally improved as the software changes. Refactoring is a potentially risky activity because of the possibility of introducing defects. This risk is mitigated when the software has a comprehensive suite of automated tests.

Repeatable Build and Deploy

Every software team has some kind of method for building and deploying their software. I wanted to emphasize repeatability because that is the essential aspect of this practice. A build and deploy process that you can repeat consistently across team members, across environments, and across different versions of the software is critical to successfully shipping your software or deploying it into production. This practice only works effectively when version control is used to define the official version of the software being built and deployed.

A related aspect of this practice is keeping your active development mainline in good shape, which I define as having the code compile and pass all unit tests. This allows developers to reliably check out, build, and work with the latest changes, which helps avoid integration hell.

For the last while I have been involved with software deployment and I have written a couple of articles on the topic: Architecting for Deployability and Designing for Deployability.

I have encountered teams using ad-hoc build and deploy processes. I have observed such teams often experiencing production failures due to the incorrect implementation of changes into the production environment.

Communication

I see the other practices frequently mentioned by software consultants and software development books, but the practice of communication seems to receive less attention. However, I consider it equally essential. Frequent and open communication between team members, across teams, and across layers of the organization are critical to producing good software. Successful large open source software projects are excellent examples of communication in action: they have multiple mailing lists, standards for discussing changes and informing others of patches, and defect or task tracking tools for communicating issues and task assignments.

When communication is lacking or ineffective within a team, then I have observed work being done by one team member conflicting with or disrupting the work of other team members. Poor communication between teams can be more serious: teams working on interdependent applications or components need to keep the other teams informed and involved when the interfaces between them change. I have seen failure to do so lead to schedule slippages due to the additional work being identified too late, and even production failures.

You may feel that some of these practices are so obvious that they are not worth discussing. For every one of these practices, however, I have encountered individuals who did not see the value of the practice or who only adopted it grudgingly. If you or your team does not follow one of these practices, I strongly recommend giving it a try.