<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Basil Vandegriend: Professional Software Development &#187; unit testing</title>
	<atom:link href="http://www.basilv.com/psd/blog/category/unit-testing/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.basilv.com/psd</link>
	<description></description>
	<lastBuildDate>Wed, 16 May 2012 13:28:15 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.3</generator>
		<item>
		<title>Reviewing Automated Test Code</title>
		<link>http://www.basilv.com/psd/blog/2011/reviewing-automated-test-code</link>
		<comments>http://www.basilv.com/psd/blog/2011/reviewing-automated-test-code#comments</comments>
		<pubDate>Wed, 05 Jan 2011 13:55:35 +0000</pubDate>
		<dc:creator>Basil Vandegriend</dc:creator>
				<category><![CDATA[unit testing]]></category>
		<category><![CDATA[code review]]></category>
		<category><![CDATA[quality]]></category>

		<guid isPermaLink="false">http://www.basilv.com/psd/?p=588</guid>
		<description><![CDATA[Software engineering research and my own experience have taught me the value of code reviews and automated tests, especially automated unit tests written using test-driven development (TDD). In the past my general pattern when doing code reviews has generally been to focus on production code and do a more minimalistic review of the unit test [...]]]></description>
			<content:encoded><![CDATA[<p>Software engineering research and my own experience have taught me the value of <a href="http://www.basilv.com/psd/blog/2007/strategies-for-effective-code-reviews">code reviews</a> and automated tests, especially <a href="http://www.basilv.com/psd/blog/2006/how-to-write-good-unit-tests">automated unit tests</a> written using <a href="http://www.basilv.com/psd/blog/2009/test-driven-development-benefits-limitations-and-techniques">test-driven development</a> (TDD). </p>
<p>In the past my general pattern when doing code reviews has generally been to focus on production code and do a more minimalistic review of the unit test code. But over the past year, I have been seeing indications that this might be a mistake. One respected colleague even advised me to reverse my priorities and focus primarily on the tests. So why would reviewing test code be so important? </p>
<h3>Reasons to review test code</h3>
<p>Research by Caper Jones indicates that errors in test cases are quite common - as high as 20%. In other words, one in five test cases has something wrong. Caper was likely referring to manual test scripts rather than automated test code, but there's no reason to expect the act of automation to magically fix all such errors. I recently reviewed a set of unit tests for a class and found one test that made no sense - it appeared to be verifying behavior against the wrong expected result. When I raised the issue with the developer, they confirmed that the test was wrong. Fixing the test revealed a defect in the production code which they then promptly fixed. This was the same production code that I had already carefully reviewed.</p>
<p>Automated tests can have other issues besides simply being incorrect. Tests can be incomplete, weak, fragile, or hard to read and maintain. </p>
<p>Incomplete tests fail to test certain functionality. Reviewing the results of a code coverage tool is a very easy way to identify gaps in coverage of the existing production code. A more careful review of the actual tests is needed, however, to ensure that all required functionality is actually implemented since code coverage cannot identify gaps when production code is missing. </p>
<p>Weak tests have a low likelihood of detecting defects. One example is tests for trivial getters and setters - such tests are so weak that I generally consider them useless clutter and delete them when I see them in a review. Test weakness is really a continuum from virtually useless, as per my example above, to extremely powerful. Part of reviewing tests is to identify when the power of the test can be improved. A while ago I had to change a business rule to use less-than-or-equals logic rather than less-than logic as originally specified. I confirmed there were tests covering this rule, then changed the logic and re-ran the tests expecting them to fail, but to my surprise they all passed. Upon investigating, I discovered that the tests improperly exercised the boundary condition - they tested a value below the boundary and above the boundary, but not on the boundary. So the equal-to-boundary case, which was the logic I was changing, was not tested. This prompted me to review the tests for other similar functionality, where I found the same pattern of weak boundary tests, a few of which had led to defects escaping detection. If we had been more diligent reviewing the unit tests when this functionality was first written we could have avoided these problems.</p>
<p>Fragile tests have a high likelihood of requiring updates when production code is changed. In other words, changes are likely to cause such tests to start failing. Having too many fragile tests can lead to maintenance issues down the road - developers may choose to delete such failing fragile tests rather than fixing them, or worse might choose to stop running the tests altogether. Martin Fowler termed this <a href="http://martinfowler.com/bliki/TestCancer.html">test cancer</a>. Reviews by individuals experienced in maintaining unit tests are the only way I know of detecting and minimizing fragile tests.</p>
<p>Test code needs to be clean - easy to read and maintain - just like production code. In fact, many proponents of automated tests argue that well-written tests can function as documentation highlighting what the production code is supposed to do and providing examples of how to use specific APIs. In my experience, using tests as documentation only works when the test code is clean, which generally requires reviews to ensure it happens consistently.</p>
<h3>Incorrect perceptions regarding reviewing test code</h3>
<p>Given all these compelling arguments for reviewing unit tests, why have I not reviewed tests as much in the past? The 'excuses' I have used seem to be due to incorrect perceptions on my part.</p>
<p>As an architect, I often do what I call architect-level reviews, focusing on the overall structure of the production code and non-functional attributes. I am often not familiar with the detailed functional requirements the code is trying to satisfy. So I don't bother looking at the details of the test code, since I usually can't confirm if the expected results are actually what the requirements specify. But wait - if I am able to still effectively review the production code, then I can perform the same type of review for the test code.</p>
<p>I focus first on production code due to my underlying belief that correct and clean production code is more important than test code. In one sense, this is true. But there are other viewpoints. One view is that if it isn't tested, it doesn't work. Another view is that the long-term vitality of the production code over multiple release depends on a solid, well-written suite of automated tests. </p>
<p>Related to the prior error in perception is the issue of schedule and/or budget constraints. Given limited time or effort, where should I allocate my time for a review? If my priority is production code, then I focus on that and neglect the test code. But this very question contains a flawed assumption: it assumes I should sacrifice the thoroughness of the review for other constraints. But the defects I fail to detect as a result of this sacrifice will likely get caught and fixed later at much greater expense. Reviews are one of the most effective means of defect removal - they are usually twice as effective as most kinds of testing at finding defects and are half as costly to fix the defects that are found. </p>
<h3>Conclusion</h3>
<p>While I hope this article helps convince you of the importance of reviewing test code, I wrote it just as much for myself to reinforce the importance of doing these reviews. Here are some pointers for getting started:</p>
<ul>
<li>Use a code coverage tool like Cobertura to review how well the tests cover the production code.</li>
<li>Review the tests first, before reviewing the production code.</li>
<li>Use the following questions as a guide to reviewing test code:
<ul>
<li>Are the requirements fully tested? Are there any omissions of functionality?</li>
<li>Are the tests weak or fragile?</li>
<li>Are the tests readable and the code clean (e.g. no duplication)?</li>
</ul>
</li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://www.basilv.com/psd/blog/2011/reviewing-automated-test-code/feed</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Java Unit Testing Tutorial: Write Your First Unit Test</title>
		<link>http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-write-your-first-unit-test</link>
		<comments>http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-write-your-first-unit-test#comments</comments>
		<pubDate>Fri, 16 Jan 2009 13:33:38 +0000</pubDate>
		<dc:creator>Basil Vandegriend</dc:creator>
				<category><![CDATA[unit testing]]></category>

		<guid isPermaLink="false">http://www.basilv.com/psd/?p=240</guid>
		<description><![CDATA[In this section of the Java Unit Testing Tutorial you will write your first unit test and get it to execute successfully. Tutorial Introduction Previous: Initial Setup Determine What to Test Before you can write your first unit test you need some code to test. If you are trying this in your own project, then [...]]]></description>
			<content:encoded><![CDATA[<p>In this section of the <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial">Java Unit Testing Tutorial</a> you will write your first unit test and get it to execute successfully.</p>
<p><a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial">Tutorial Introduction</a> <strong>Previous:</strong> <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-initial-setup">Initial Setup</a></p>
<h3>Determine What to Test</h3>
<p>Before you can write your first unit test you need some code to test. If you are trying this in your own project, then select a class with a method that needs unit testing. How do you know what code needs testing? The general criteria is this: when you look at the code for the class, if there is any logic which you are uncertain will work in all scenarios, then you should write unit tests for that logic. To be specific, one-line methods like trivial getter and setter methods do not need testing – it is obvious that they will work. Most methods longer than a few lines of code and especially those with conditionals or loops usually warrant unit tests. </p>
<p>You must be wary of false confidence in your code. You need to be one hundred percent confident that the code works as intended across all scenarios – including bad input, system failure, null references, etc. - in order to decide it does not need a unit test.</p>
<p>As you are choosing your class to test, it will be helpful for this first time to choose a class with minimal dependencies: the class should require a minimum (ideally none) of other classes / objects to be supplied for the class to be used.</p>
<p>In the remainder of this section I will refer to the class and method you have chosen as class <code>YourClass</code> located in package <code>your.package</code> and method <code>yourMethod()</code>. </p>
<p>To provide a concrete example I will be testing the class <code>ParserHelper</code> from the example project <code>UnitTestingTutorial</code>. The code for this class is shown below:</p>
<pre class="prettyprint">
// Copyright 2009 by Basil Vandegriend.  All rights reserved.
package com.basilv.unittestingtutorial.writeyourfirsttest;

/**
 * Helps parse a set of tokens. Maintains an internal index into the
 * supplied tokens to determine which token all the methods will act on.
 * This index must be advanced by the caller.
 */
public class ParserHelper
{
  static final String DEFINITION_SEPARATOR = "->";

  private String[] tokens;

  private int currentTokenIndex;

  public ParserHelper(String[] tokens) {
    this.tokens = tokens;
  }

  public void nextToken() {
    currentTokenIndex++;
  }

  public String extractTaskName() {
    if (!haveMoreTokens()) {
      throw createParseException("Expected task name but no further input supplied.");
    }
    return currentToken();
  }

  public String extractRollupGroup() {
    if (!haveRollupGroup()) {
      throw createParseException("Expected a rollup group of the form '['<group name>']'.");
    }
    String token = currentToken();
    String group = token.substring(1, token.length() - 1);
    if (group.length() == 0) {
      throw createParseException("An empty rollup group '[]' was specified.");
    }
    return group;
  }

  public boolean haveRollupGroup() {
    if (!haveMoreTokens()) {
      return false;
    }
    String token = currentToken();

    return (token.startsWith("[") &#038;& token.endsWith("]"));
  }

  public void expectSeparator() {
    if (!haveSeparator()) {
      throw createParseException("Expected separator '"
        + DEFINITION_SEPARATOR + "'.");
    }
  }

  private boolean haveSeparator() {
    if (!haveMoreTokens()) {
      return false;
    }
    return DEFINITION_SEPARATOR.equals(currentToken());
  }

  private boolean haveMoreTokens() {
    return currentTokenIndex < tokens.length;
  }

  private String currentToken() {
    return tokens[currentTokenIndex];
  }

  private RuntimeException createParseException(
    String parseErrorMessage) {
    int tokenNumber = currentTokenIndex + 1;
    String message = "Error parsing token #" + tokenNumber
      + ":" + parseErrorMessage;

    return new RuntimeException(message);
  }
}
</pre>
<p>This class has a number of methods. Which one should I test? The <code>nextToken</code> method is trivially correct so I skip it. The <code>extractTaskName</code> method is almost simple enough to skip, but I resist the temptation and choose it. (Because the method is simple it will be easy and quick to test.)</p>
<h3>Create the Unit Test Skeleton</h3>
<p>Before you can write the actual unit test code to invoke the method you chose and verify the results, you must create the skeleton code that JUnit needs to execute the unit test. This skeleton test code follows a standard pattern which is explained in the following steps:</p>
<ol>
<li>Create a test class for class <code>YourClass</code>. The test class will be located in the same package <code>your.package</code> but under the source directory "src/test" instead of "src/main". Name the class <code>YourClassTest</code>. If the class you are planning to test is located on the file system at "src/main/your/package/YourClass.java" you will now have a test class located at "src/test/your/package/YourClassTest.java".</li>
<li>Populate the test class using the following template. Include the import statements. You will need to change the name of the class in the template to <code>YourClassTest</code>.
<pre class="prettyprint">
import static org.junit.Assert.*;
import org.junit.*;

public class TemplateClassTest
{
  @Before
  public void setUp() throws Exception {

  }

  @After
  public void tearDown() throws Exception {

  }

  @Test
  public void templateMethod() throws Exception {
    fail("TODO: Write this test.");
  }

}
</pre>
<p>This template provides a single test method identified by the <code>@Test</code> annotation. JUnit will treat this method as a test and execute it as part of the project's suite of tests. </p>
<p>The <code>@Before</code> and <code>@After</code> annotations indicate methods that are to be invoked before and after the test method. Collectively these two methods represent what is called a test fixture, and are used to create a common set of objects shared across multiple tests.
</li>
<li>Rename method <code>templateMethod()</code> to <code>yourMethod()</code>. The test code you will write will go in this method.
</li>
<li>Run the unit tests for the project to confirm the test skeleton was created properly. The <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-initial-setup">Initial Setup</a> section of the tutorial explains how to do this in detail, but as a quick reminder open the menu for the Run icon in the tool bar and choose the JUnit entry named after your project. The tests should now run (without the warning message about having no tests that you encountered during initial setup).</li>
<li>The results of running JUnit tests are displayed within an Eclipse view titled "JUnit". See the image below. (Your JUnit view may be laid out differently depending on your screen and options, but will contain the same information.) The most important element of the view is the horizontal red bar near the top, which is a striking visual indicator of the overall status of the tests: red indicates that at least one test was unsuccessful, whereas green is displayed only if all the tests are successful. Above the bar are the summary statistics that indicate the number of tests that ran (1 / 1), the number that encountered an error (0), and the number that failed to meet the expected results (1). Below the red bar in the first pane is the hierarchy of tests, usually filtered to only show the unsuccessful tests. The second pane shows the stack trace of the point of failure or error for the test selected in the first pane.<br />
<a href="http://www.basilv.com/psd/wp-content/uploads/2009/01/eclipsejunitviewwithfailure.png"><img src="http://www.basilv.com/psd/wp-content/uploads/2009/01/eclipsejunitviewwithfailure.png" alt="Eclipse JUnit View with failure" title="Eclipse JUnit View with failure" width="424" height="512" class="size-full wp-image-252" /></a><br />
Your single test is failing because the test method provided in the template includes the following code:
<pre class="prettyprint">fail("TODO: Write this test.");</pre>
<p>The <code>fail()</code> method is a static method from the JUnit <code>Assert</code> class that causes the test to fail if it is executed. Including this code in the template provides an automatic reminder to actually write the test as it starts out failing. </li>
</ol>
<p>Returning to the example project, since I am planning to test the class <code>ParserHelper</code> I need to create a test class <code>ParserHelperTest</code>. The skeleton code for this test class is located below:</p>
<pre class="prettyprint">
// Copyright 2009 by Basil Vandegriend.  All rights reserved.
package com.basilv.unittestingtutorial.writeyourfirsttest;

import static org.junit.Assert.*;
import org.junit.*;

public class ParserHelperTest
{
  @Before
  public void setUp() throws Exception {

  }

  @After
  public void tearDown() throws Exception {

  }

  @Test
  public void extractTaskName() throws Exception {
    fail("TODO: Write this test.");
  }
}
</pre>
<p>Because I plan to test the method <code>extractTaskName</code> on <code>ParserHelper</code>, I have given the test method the same name.</p>
<h3>Write the Test Code</h3>
<p>You are now ready to write the actual test code that will exercise the class you are testing and verify the results. While the specifics will of course vary from class to class and method to method, there are three common phases that must be performed in each case: setup, execution, and verification.</p>
<ol>
<li><strong>Setup:</strong> Create the object to be tested and configure its initial state prior to being tested. This includes configuring any required dependencies, which includes parameters to the method being tested.</li>
<li><strong>Execution:</strong> Invoke the method being tested and capture the return result.</li>
<li><strong>Verification:</strong> Confirm that the method performed the expected actions. This is done by invoking appropriate methods of the JUnit <code>Assert</code> class such as the <code>fail()</code> method introduced above. If an expectation fails then the JUnit method will throw an special exception indicating the test has failed. Execution of the test method will stop and JUnit will record a failure for the test. If at any point in the test method any other exception is unexpectedly thrown (i.e. not caught within the test method) JUnit will record an error for the test.</li>
</ol>
<p>To demonstrate how to implement these steps I continue the example from above to test the <code>ParserHelper.extractTaskName()</code> method.</p>
<ol>
<li><strong>Setup:</strong> I need to create an instance of <code>ParserHelper</code>, which requires  an array of tokens to be supplied to its constructor. I chose to supply a single token, which will be the task name that I expect to have returned from the method.
<pre class="prettyprint">
    String expectedTaskName = "test";
    ParserHelper helper = new ParserHelper(new String[] { expectedTaskName });
</pre>
</li>
<li><strong>Execution:</strong> I invoke the method under test and record the actual result.
<pre class="prettyprint">
 String actualTaskName = helper.extractTaskName();
</pre>
</li>
<li><strong>Verification:</strong> I use the JUnit <code>assertEquals</code> method to verify that the actual task name returned matches what I expect.
<pre class="prettyprint">
    assertEquals(expectedTaskName, actualTaskName);
</pre>
</li>
</ol>
<p>The completed test method is shown below:</p>
<pre class="prettyprint">
  @Test
  public void extractTaskName() throws Exception {
    // Setup
    String expectedTaskName = "test";
    ParserHelper helper = new ParserHelper(new String[] { expectedTaskName });

    // Execution
    String actualTaskName = helper.extractTaskName();

    // Verification
    assertEquals(expectedTaskName, actualTaskName);
  }
</pre>
<p>This concludes the tutorial. There is, however, still much more to learn about unit testing. Check here regularly or subscribe to this site's <a href="http://www.basilv.com/psd/feed">feed</a> to find out when new sections are added to this tutorial. In the meantime, here are some links to additional resources on unit testing:</p>
<ul>
<li><a href="http://junit.sourceforge.net/doc/cookbook/cookbook.htm">JUnit Cookbook</a></li>
<li><a href="http://junit.sourceforge.net/doc/faq/faq.htm">JUnit Frequently Asked Questions</a></li>
<li><a href="http://www.basilv.com/psd/blog/2006/how-to-write-good-unit-tests">My article on how to write good unit tests by avoiding common pitfalls</a></li>
</ul>
<p><a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial">Tutorial Introduction</a> <strong>Previous:</strong> <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-initial-setup">Initial Setup</a> </p>
]]></content:encoded>
			<wfw:commentRss>http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-write-your-first-unit-test/feed</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
		<item>
		<title>Java Unit Testing Tutorial: Initial Setup</title>
		<link>http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-initial-setup</link>
		<comments>http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-initial-setup#comments</comments>
		<pubDate>Fri, 16 Jan 2009 13:33:01 +0000</pubDate>
		<dc:creator>Basil Vandegriend</dc:creator>
				<category><![CDATA[unit testing]]></category>

		<guid isPermaLink="false">http://www.basilv.com/psd/?p=232</guid>
		<description><![CDATA[In this section of the Java Unit Testing Tutorial you will install JUnit, the unit testing framework you will use for unit testing, configure Eclipse to run your project's tests, and finally configure your project for the unit test code you will write. Previous: Tutorial Introduction Next: Writing Your First Unit Test Install JUnit Follow [...]]]></description>
			<content:encoded><![CDATA[<p>In this section of the <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial">Java Unit Testing Tutorial</a> you will install JUnit, the unit testing framework you will use for unit testing, configure Eclipse to run your project's tests, and finally configure your project for the unit test code you will write.</p>
<p><strong>Previous:</strong> <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial">Tutorial Introduction</a> <strong>Next:</strong> <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-write-your-first-unit-test">Writing Your First Unit Test</a></p>
<h3>Install JUnit</h3>
<p>Follow these steps to install JUnit:</p>
<ol>
<li><a href="http://sourceforge.net/project/showfiles.php?group_id=15278">Download Junit</a>. You just need the individual jar file, but you may want to grab the zip file that also includes the source code and documentation..</li>
<li>Place the junit jar file on the classpath for your project:
<ol>
<li>Create a directory called "libs" under your project root directory.</li>
<li>Copy the file "junit-4.5.jar" you downloaded into this directory.</li>
<li>Ensure that the directory and jar file are visible within Eclipse. If not, right-click on the project and select "Refresh" to synchronize Eclipse with the underlying file system.</li>
<li>Within Eclipse navigate to the junit jar file, right-click on it, and choose "Build Path | Add to Build Path".
</ol>
</li>
</ol>
<h3>Configure Eclipse</h3>
<p>Follow these steps to configure Eclipse to run your project's tests:</p>
<ol>
<li>Within Eclipse right-click on the project and choose "Run As | JUnit Test". A dialog should appear with the message "No tests found with test runner 'JUnit 4.0'.". This message appears because we have not yet created any unit tests. You will fix that in the next section of this tutorial. Despite the message, performing this action created the run configuration.</li>
<li>Right-click again on the project and choose "Run As | Run Configurations...". This will display the "Run Configurations" dialog box that allows you to edit and create run configurations. On the left side is a tree listing all the run configurations. Under the node "JUnit", click on the run configuration with a name corresponding to the name of your project. (This will be "UnitTestingTutorial for our example.) Note that under the tab "Test" on the right it indicates that this configuration will "Run all tests in the selected project".</li>
<li>Select the tab "Common", and within the section "Save As" choose the option "Shared file". This creates a run configuration file within your project which you can check into version control and thus make available to the rest of your team. By default this launch file will be saved to your project's root directory. Within the section "Display in favorites menu" select the option "Run". Click the button "Apply" to save your changes, then select "Close".</li>
<li>Check your project's root directory to confirm that the run configuration file was created. It will be named the same as your project name in Eclipse with an extension of ".launch".</li>
<li>Within Eclipse expand the favorites menu of the Run button in the Toolbar as per the image below. The top entry of this menu should correspond to the run configuration you just made - it should be named the same as your project with a JUnit icon. Now ignore the menu and click the Run button. This will run the top entry of the menu, executing the project's unit tests. The message that no tests were found should appear again.</li>
</ol>
<p><a href="http://www.basilv.com/psd/wp-content/uploads/2009/01/eclipserunfavoritesmenu.png"><img src="http://www.basilv.com/psd/wp-content/uploads/2009/01/eclipserunfavoritesmenu.png" alt="Eclipse Run Favorites Menu" title="Eclipse Run Favorites Menu" width="433" height="232" class="alignnone size-full wp-image-237" /></a> </p>
<h3>Configure Your Project</h3>
<p>Follow these steps to configure your project for the unit test code you will write:</p>
<ol>
<li>Create the directory structure "src/test" under the root directory of your project. This is where the unit test code you will write will go. In Eclipse right-click on this "test" directory and select "Build Path | Use as Source Folder".</li>
<li>Create the directory structure "src/main" under the root directory of your project. This is where the actual project code should go. In Eclipse right-click on this "main" directory and select "Build Path | Use as Source Folder". If your existing project has the source code under a different directory, move it under "src/main", right click on the other directory and choose "Build Path | Remove from Build Path".</li>
</ol>
<p>This concludes the initial setup. You can now execute tests for your project, but without any tests this is not too interesting. Fortunately you are now ready to create your first unit test and this is the topic of the next section of this tutorial.</p>
<p><strong>Previous:</strong> <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial">Tutorial Introduction</a> <strong>Next:</strong> <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-write-your-first-unit-test">Writing Your First Unit Test</a></p>
]]></content:encoded>
			<wfw:commentRss>http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-initial-setup/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Java Unit Testing Tutorial</title>
		<link>http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial</link>
		<comments>http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial#comments</comments>
		<pubDate>Fri, 16 Jan 2009 13:32:24 +0000</pubDate>
		<dc:creator>Basil Vandegriend</dc:creator>
				<category><![CDATA[unit testing]]></category>
		<category><![CDATA[coding]]></category>
		<category><![CDATA[Java]]></category>
		<category><![CDATA[software development]]></category>

		<guid isPermaLink="false">http://www.basilv.com/psd/?p=227</guid>
		<description><![CDATA[This tutorial explains how to set up and write automated unit tests in Java using JUnit 4, a widely used java unit testing framework, Eclipse 3, an open source integrated development environment (IDE), and Java SE 5 or later. This tutorial is intended to be an introduction to unit testing aimed at developers who have [...]]]></description>
			<content:encoded><![CDATA[<p>This tutorial explains how to set up and write automated unit tests in Java using <a href="http://junit.org/">JUnit 4</a>, a widely used java unit testing framework, <a href="http://www.eclipse.org/">Eclipse 3</a>,  an open source integrated development environment (IDE)</a>, and <a href="http://java.sun.com/">Java SE 5</a> or later. This tutorial is intended to be an introduction to unit testing aimed at developers who have limited or no experience with it. A working knowledge of Java (including Java language features new to version 5) and Eclipse is assumed. The vision for this tutorial is to enable you to adopt unit testing as part of your Java development activities by explaining how to setup, write, and use unit tests.</p>
<p>What are automated unit tests? They are a type of testing performed by developers to ensure that the code they write works as intended. They are automated: tests are expressed as test code that automatically invokes the code to be tested and verifies that the results of execution match the developer's expectations. The scope of testing is individual units of functionality within the code base, such as a single class or method.</p>
<p>This tutorial is oriented towards beginners and therefore is prescriptive. It provides a single step-by-step method for setting up and writing unit tests, usually without explaining the reasons behind why a particular method was used or what some alternative options may be.</p>
<p>This tutorial assumes you already have a Java project with some code (maybe only a single class) set up in the Eclipse IDE. As you go through the tutorial, you should apply the instructions to your project. The tutorial uses a sample project called <code>UnitTestingTutorial</code> to provide concrete examples when appropriate. Use the link below to download this project.</p>
<p><a href="http://www.basilv.com/psd/software-files/UnitTestingTutorialExampleProject-1.0.zip" class="download">Download the Unit Testing Tutorial Example Project v1.0</a></p>
<p>This tutorial is divided into a number of sections, each of which is presented as a separate page:</p>
<ul>
<li><a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-initial-setup">Section 1: Initial Setup</a>: How to install JUnit and configure your Java project to use it.</li>
<li><a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-write-your-first-unit-test">Section 2: Write Your First Unit Test</a>: Create your first automated unit test by writing test code using JUnit and watch the test pass.</li>
</ul>
<p>I plan to add additional sections to this tutorial over time. If there is particular content you are interested in let me know in the comments below.</p>
<p><strong>Next:</strong> <a href="http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial-initial-setup">Section 1: Initial Setup</a></p>
]]></content:encoded>
			<wfw:commentRss>http://www.basilv.com/psd/blog/2009/java-unit-testing-tutorial/feed</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>How to Write Good Unit Tests</title>
		<link>http://www.basilv.com/psd/blog/2006/how-to-write-good-unit-tests</link>
		<comments>http://www.basilv.com/psd/blog/2006/how-to-write-good-unit-tests#comments</comments>
		<pubDate>Thu, 23 Feb 2006 15:00:43 +0000</pubDate>
		<dc:creator>Basil Vandegriend</dc:creator>
				<category><![CDATA[unit testing]]></category>
		<category><![CDATA[design]]></category>

		<guid isPermaLink="false">http://www.basilv.com/psd/blog/2006/how-to-write-good-unit-tests</guid>
		<description><![CDATA[My previous post on How to Create Maintainable Software talked about the importance of automated testing. Working with automated test suites over the years, I've found that creating good tests is a skill that needs to be developed. I've seen many cases of unit tests that I'd consider poor or bad due to a number [...]]]></description>
			<content:encoded><![CDATA[<p>My previous post on <a href="/psd/blog/2006/how-to-create-maintainable-software">How to Create Maintainable Software</a> talked about the importance of automated testing. Working with automated test suites over the years, I've found that creating good tests is a skill that needs to be developed. I've seen many cases of unit tests that I'd consider poor or bad due to a number of reasons: tests that exercise simple code unlikely to break, tests that are brittle and fail due to the most minor of change, such as different data in the database or a minor user interface change, or the tests depend on data in the database that is poorly managed. In this article I will provide examples of such bad tests and show how they can be improved.</p>
<p>This article assumes you are familiar with unit testing in Java using <a href="http://www.junit.org">JUnit</a>. If not, check out the documentation on the <a href="http://www.junit.org">JUnit website</a>. A good introductory book on the topic is <a href="http://www.amazon.ca/exec/obidos/redirect?link_code=as2&#038;path=ASIN/0974514012&#038;tag=basilvandegri-20&#038;camp=15121&#038;creative=330641">Pragmatic Unit Testing in Java with JUnit</a><img src="http://www.assoc-amazon.ca/e/ir?t=basilvandegri-20&#038;l=as2&#038;o=15&#038;a=0974514012" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /> by Andrew Hunt and Dave Thomas. The back of the book has a summary page that does a great job highlighting the main concepts explained in the book, and is worth looking at even if you have experience with unit testing.</p>
<p>In each of the following sections I discuss a particular category of bad or poor tests, provide an example, and explain how such tests can be improved.</p>
<h3>Testing trivial code</h3>
<p>Unit tests should test those segments of code that are likely to have defects when first developed, or are likely to have defects introduced when changed. Like all software development activities, there is a cost benefit analysis that can be applied to writing unit tests. For normal enterprise business software, it is not worthwhile to test trivial code.</p>
<p>Typical examples of trivial code in Java include simple getter and setter methods for properties and simple constructors. </p>
<pre class="prettyprint">
class User {
  private String name;

  public String getName() {
    return name;
  }

  public void setName(String newName) {
    name = newName;
  }
}

class UserTest extends TestSuite {
  public void testNameProperty() {
    User user = new User();
    assertNull(user.getName());

    String testName = "test";
    user.setName(testName);
    assertEquals(testName, user.getName());
  }
}
</pre>
<p>When I see tests such as these, I simply delete them. It is unlikely for simple getter/setter methods to change in the future. The test doesn't improve my understanding of the code and just takes up space, hiding the more useful tests. </p>
<h3>Using hardcoded values when checking test results</h3>
<p>I sometimes see tests that hardcode a 'magic' value when checking the results of some operation. This magic value is often separately hardcoded in the application code being tested. When the value is changed in the application code, the test is guaranteed to fail. </p>
<pre class="prettyprint">
class CustomerWebController {

  public String doOperationReturningNextPage(UserInput input) {
    // some random logic...
    return "newCustomer.jsp";
  }
}

class CustomerWebControllerTest extends TestSuite {
  public void testDoOperation() {
    CustomerWebController controller = new CustomerWebController();
    UserInput input = getInputForNewCustomer();
    String result = controller.doOperationReturningNextPage(input);
    assertEquals("newCustomer.jsp", result);
  }
}
</pre>
<p>The fix for this is simply the application of the DRY principle: Don't Repeat Yourself (from the book <a href="http://www.amazon.ca/exec/obidos/redirect?link_code=as2&#038;path=ASIN/020161622X&#038;tag=basilvandegri-20&#038;camp=15121&#038;creative=330641">The Pragmatic Programmer: From Journeyman to Master</a><img src="http://www.assoc-amazon.ca/e/ir?t=basilvandegri-20&#038;l=as2&#038;o=15&#038;a=020161622X" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /> by Andrew Hunt and David Thomas). When you go to use a magic value more than once, define it as a constant (or method) and then refer to that constant (or method).</p>
<p>To improve the example, we define a constant for the "newCustomer.jsp" value.</p>
<pre class="prettyprint">
class CustomerWebController {

  public static final NEW_CUSTOMER_PAGE = "newCustomer.jsp";

  public String doOperationReturningNextPage(UserInput input) {
    // some random logic...
    return NEW_CUSTOMER_PAGE;
  }
}

class CustomerWebControllerTest extends TestSuite {
  public void testDoOperation() {
    CustomerWebController controller = new CustomerWebController();
    UserInput input = getInputForNewCustomer();
    String result = controller.doOperationReturningNextPage(input);
    assertEquals(CustomerWebController.NEW_CUSTOMER_PAGE, result);
  }
}
</pre>
<h3>Being too dependent on specific test data</h3>
<p>There is some debate whether unit tests should involve the database (purists tend to argue not), but in practise this is quite common and serves a useful purpose. However, problems often occur because such tests are overly dependent on specific data in the database. As new tests are written and new test data is added, this can cause existing, unrelated tests to fail.</p>
<pre class="prettyprint">
class CustomerDataAccess {
  public List<Customer> findCustomers(CustomerCriteria criteria) {
    // Logic to query database based on criteria
    return customersFound;
  }

class CustomerDataAccessTest extends TestSuite {
  public void testFindCustomers {
    CustomerDataAccess customerDataAccess = new CustomerDataAccess();
    CustomerCriteria criteria = new CustomerCriteria();
    String firstNameToFind = "Bob";
    criteria.firstNameEquals(firstNameToFind);
    List<Customer> results = customerDataAcess.findCustomers(criteria);
    assertEquals(2, results.size());
  }
}
</pre>
<p>This example is based on real code that I came across. The problem is that the test expects there to be only two customers with a first name of 'Bob', which must have been the case when the test was first written and executed.  However at some later date, working on some unrelated piece of code, a developer could add another customer named Bob, and suddenly this test will fail. To fix this type of situation in general, you need to minimize the number of assumptions you make about the test data. This reduces the level of coupling between the test and the data, which makes either easier to change independent of the other.</p>
<p>To improve this particular example we simply need to change the test to check that each result matches the criteria we specified.</p>
<pre class="prettyprint">
class CustomerDataAccessTest extends TestSuite {
  public void testFindCustomers {
    CustomerDataAccess customerDataAccess = new CustomerDataAccess();
    CustomerCriteria criteria = new CustomerCriteria();
    String firstNameToFind = "Bob";
    criteria.firstNameEquals(firstNameToFind);
    List<Customer> results = customerDataAcess.findCustomers(criteria);
    for (Customer customer : results) {
      assertEquals(firstNameToFind, customer.getFirstName());
    }
  }
}
</pre>
<h3>Slowly executing tests</h3>
<p>One problem with having unit tests involve the database is that such tests tend to run slower than in-memory tests. As the suite of tests grow, it is important to minimize the time required to run the entire suite. Ideally you should execute the entire test suite before checking changes into the version control repository. The longer it takes to run the tests, the less likely you are to do this, and the more likely you are to check in changes that break the tests, which defeats the entire purpose of having them. A rule of thumb I use is that all tests should execute within about five minutes.</p>
<p>When I joined one project, the unit tests took almost two hours to run. When I had the opportunity I analyzed the performance of the tests to determine which tests were taking the longest and why. I discovered several issues, all dealing with database access. The most significant issue was the setup and teardown being done for the persistence tests. Due to the design of the application, any creating or updating of records required a User object to be supplied, and in order to meet foreign key constraints, this User object must be defined in the database. Each persistence test created a new user object (plus all associated objects, when needed) in the database in the setup phase, and deleted all these new objects from the database in the teardown phase. Since in JUnit the setup and teardown are performed before and after each test method, this leads to a lot of database calls. </p>
<pre class="prettyprint">
class UserTest extends TestSuite {
  public User createTestUser() {
    // Create test user in database.
    return user;
  }

  public void deleteTestUser(User user) {
    // Delete test user from database.
  }
}

class CustomerTest extends TestSuite {
  private User user;
  public void setUp() {
    user = UserTest.createTestUser();
  }

  public void tearDown() {
    UserTest.deleteTestUser(user);
  }

  public void testCreating() {
    // Test code ...

    // Whether or not this method needs the user object,
    // it is created before the method is called, and deleted afterwards.
  }

  public void testUpdating() {
    // Test code ...

    // Whether or not this method needs the user object,
    // it is created before the method is called, and deleted afterwards.
  }

  public void testNameValidation() {
    // Test code ...

    // Whether or not this method needs the user object,
    // it is created before the method is called, and deleted afterwards.
  }

}
</pre>
<p>To improve the situation, I introduced special cached test users to use for these tests, which would only need to be loaded once for all the tests. This vastly improved the performance of the persistence tests, but did have a cost: these tests were now more dependent on specific test data, which I specifically mentioned already as something to avoid as much as possible. The general principle I use to minimize the risk of working with shared data is to have shared data be read-only. Tests using shared data are not allowed to modify that data. If a test needs to modify such data, it has to create its own copy. </p>
<p>You cannot easily delete the cached test user from the database within any specific test, since a subsequent test may want to make use of it. This means that test users will continually be created in the database. To avoid this, the initial creation of the test user can look for the existence of a special test user (i.e. via a special name), and if found use that instead of creating a new instance, but this further increases the risk of that user record being modified in the database and breaking tests as a result. A better option would be to perform a special setup and teardown operation once for a set of tests, but JUnit does not easily provide this functionality. (<a href="http://testng.org">TestNG</a> is a newer Java testing framework with features such as this.)</p>
<p>After I finished my performance optimizations, the tests for this project executed in about twelve minutes - still not great, but a big improvement from two hours.</p>
<pre class="prettyprint">
class UserTest extends TestSuite {

  private static User cachedTestUser;

  public static synchronized User getCachedTestUser() {
    if (cachedTestUser == null) {
      cachedTestUser = loadTestUserCreatingIfNotExist();
    }
    return cachedTestUser;
  }

}

class CustomerTest extends TestSuite {
  private User user;
  public void setUp() {
    user = UserTest.getCachedTestUser();
  }

  public void testCreating() {
    // Test code ...
  }

  public void testUpdating() {
    // Test code ...
  }

  public void testNameValidation() {
    // Test code ...
  }
}
</pre>
<p>Like any skill, the ability to write good unit tests takes effort to develop. I try to hone my test-writing skill by paying attention to a couple situations:</p>
<ol>
<li>When I encounter a defect and identify the cause, I always aim to write a test to prevent the defect from reoccuring. I also ask myself why that test didn't exist in the first place, or why it missed the defect.</li>
<li>When I encounter brittle tests (tests that unexpectedly fail due to unrelated changes), I ask myself how those tests could have been written differently to be more robust.</li>
</ol>
<p>I hope this proves helpful in improving your ability to write unit tests. If you come across other examples of bad tests, I'd be interested in hearing about them.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.basilv.com/psd/blog/2006/how-to-write-good-unit-tests/feed</wfw:commentRss>
		<slash:comments>15</slash:comments>
		</item>
	</channel>
</rss>

