Build automation has been the theme of my recent learning activities, so when I came across multiple positive references to a tool called FindBugs I decided to give it a try. My conclusion: FindBugs is worth using on all Java projects. Read below for the details.
FindBugs is a Java static analysis tool that scans compiled java code for potential defects and bad programming practices. Think of it as the Java compiler on steroids: it operates in roughly the same fashion but reports on a much larger set of errors and warnings. Static analysis makes a great complement to automated unit tests. Unit tests require effort to write, targets a specific piece of code, but can verify application-specific functionality. Static analysis requires no effort to write (beyond initial setup), targets the entire code base, but can only verify general code constructs. At least, that's the theory. How does it work in practice?
FindBugs supports a number of different ways of being used: command line, Swing GUI, integration into automated builds (i.e. via an Ant task and Hudson plugin), and Eclipse plugin. I decided to go with the plugin, and installation was as easy as adding the update site http://findbugs.cs.umd.edu/eclipse and installing it. Well, actually there were two gotchas. First, you need to be running Eclipse version 3.3 or greater for the plugin to work – RAD version 7 will not work. Second, you need to fully restart Eclipse after installing the plugin. I made the mistake of choosing the option to activate the plugin without doing a restart, which left portions of the plugin not working. It also seemed like you must bring up the Bug Explorer and Bug Details views, then restart Eclipse, in order to get those views working properly.
I used the FindBugs manual to get started. I selected my Time Reporter project to be the first guinea pig and ran FindBugs on it. I keep my code well-polished and well-tested with most Eclipse warnings turned on so I was not expecting FindBugs to turn up anything major. As I scanned the relatively small list of issues (under 30), I was surprised to see an actual defect! It actually took me a few moments of staring at it to find the problem. See the screen shot below.
It turned out this same defect occurred elsewhere in the code. FindBugs also identified cases of bad error handling that I would classify as defects: I was improperly ignoring return values from method calls like
File.mkdir(). Most of these serious issues were in test code rather than application code, which made me feel a little better (but not much).
I fixed all the issues reported by FindBugs that I agreed with, turned off one FindBugs warning I did not agree with at all, and was then left with a small number of false positives – incorrect warnings about code that was actually correct. My preferred approach to compiler warnings is to have none in the code base. This is based on the fact that if your code base has existing warnings that should be ignored then it becomes very difficult to tell when you write some code that produces a warning that should instead be fixed. People become blind to all warnings if there are always some present. Warnings should be produced as part of the developer's regular process (i.e. writing code) rather than requiring an extra step. It was trivial to configure FindBugs to run automatically, but how to get rid of the unwanted warnings?
Eclipse warnings can be eliminated by using the Java 5 annotation
@SuppressWarning, but FindBugs does not appear to support this annotation (or if it does I could not successfully determine what text must be supplied to the annotation to ignore the warning). I found some hints on the web that FindBugs does have the ability to ignore specific warnings, but it appears this feature has only been implemented in the Swing GUI and not in the Eclipse plugin. After further investigation I found a solution. I exported the current set of project warnings to an XML file, and then configured FindBugs to use that XML file as a baseline of warnings to ignore. This causes FindBugs to only report issues not in the baseline, leaving me with an empty list of FindBugs warnings – for now at least.
One annoying limitation of the FindBugs Eclipse plugin is that all configuration can only be done on a per-project basis. Unlike most other Eclipse functionality with global configuration that can be overridden on a per-project basis, FindBugs must be enabled and configured individually for each project. There were a few other wrinkles with the FindBugs Eclipse plugin. My general impression is that the plugin is still in a beta state, lagging behind the functionality offered by the FindBug Swing GUI.
Next I tried including FindBug as part of a continuous integration build running in Hudson. It was fairly easy to configure the Ant build to execute FindBugs on the project: the only snag was needing to allocate more memory for the JVM. Installing and configuring the FindBugs plugin for Hudson was likewise straight-forward, and resulted in a nice set of pages for viewing the trends and details regarding the FindBugs warnings. The big issue came when I wanted to filter out (exclude) all the warnings I did not want to fix. After much investigation and trial and error, I discovered that if I used the FindBug Swing GUI to create an exclude list, I could then configure the ant build to use that list as an exclude filter. Using the filter created by the Eclipse plugin did not seem to work. The FindBugs documentation concerning this was rather poor, but I did see a statement admitting that filter support was a bit messed up across the various FindBugs tools, and it sounds like this area will be targeted for improvement in the next year.
Despite these tooling problems, the ease with which FindBugs finds defects makes it definitely worthwhile to use on all Java projects. When I turned FindBugs loose against the code base of a reasonably-sized production application, it found 5 definite defects, 14 cases that were either potential defects or extremely bad coding style, and many other warnings (I didn't bother to count, but probably over 100) that were worth investigating and often worth fixing.
My conclusion from this is that using FindBugs is definitely worthwhile. I plan to roll it out to all my Java projects and integrate it into the automated builds so that the FindBugs results are also available from the continuous integration server. If you plan to adopt FindBugs then I recommend checking out some FindBugs tutorials. If you want to promote the use of FindBugs to your coworkers or management then I'll point out that FindBugs is a corporate standard at both Google and eBay and eBay reports that "using 2 developers to audit/review FindBugs warnings was 10 times more effective at finding P1 bugs than using two testers" (http://findbugs-tutorials.googlecode.com/files/UFIA-intro.pdf).
If you find this article helpful, please make a donation.