Evaluation of three open source code analysers

There are couple of code analysis tools avaiable out there and three of them (FindBugs, PMD and QJPro) i have evaluated and would like to elaborate some more. I will give a quick demo for each along the way and it will give you a good visualization of how each tool looks like in the real life.

FindBugs

FindBugs looks for bugs in java the programs. It can detect a variety of common coding mistakes, including thread synchronization problems, misuse of API methods and etc.

FindBugs is a static analysis tool that examines java classes or JAR files and looks for potential problems by matching class bytecodes against a list of bug patterns. With static analysis tools, we can analyze software without actually running the program. FindBugs requires JRE (or JDK) 1.4.0 or later to run. However, it can analyze programs compiled for any version of Java. FindBugs supports a plugin architecture allowing anyone to add new bug detectors, which means we can write custom detectors to find application-specific problems and it has been used for Eclipse and Java JDK project for bugs finding.

We can run FindBugs in multiple ways -- from a GUI, from a command line, using Ant, as an Eclipse plug-in, and using Maven. However, it makes more sense to run it as an integrated part of nightly build infrastructure and generate a bug report, which could be easily done by using tools like CruiseControl, Anthill, or even simple cron to fire off nightly builds and giving the results as an email. FindBugs certainly won't find all the bugs, but they'll help find some of them. FindBugs can act as a safety net and detects identified bug patterns when developers check in their codes into the cvs. You can find a demo for FindBugs here and a viewlet here.

PMD

PMD is not like FindBugs and it analyzes the java source code instead of class files, scans and looks for potential problems like:

* Possible bugs - empty try/catch/finally/switch statements
* Dead code - unused local variables, parameters and private methods
* Suboptimal code - wasteful String/StringBuffer usage
* Overcomplicated expressions - unnecessary if statements, for loops that could be while loops
* Duplicate code - copied/pasted code means copied/pasted bugs


PMD has over 175 “rules”, which are undesirable patterns of code, right out of the box. These rules are grouped into “rulesets,” such as Basic, Unused Code Design, Optimization, Strict Exception, String and StringBuffer, Naming, JUnit and Java Logging.

Not all of them will be appropriate for our project or organization. Fortunately, PMD is configurable and we can configure which rules we want enforced. To do this, we can make our own ruleset file, which states which of the rules we want executed during analysis.

PMD is integrated with Eclipse, JBuilder, IntelliJ IDEA, Maven, Ant, and JDeveoper release 10.1.2 and 10.1.3.

It can also be run on demand from the command line, which allows it to be integrated into nightly Ant/Maven builds and publish the results to a shared location such as an intranet or file server.

There are two ways we can install the PMD plugin for JDeveloper release 10.1.3. One way is to install the extension by using JDeveloper's built in "update center" functionality. Fire up JDeveloper, click "Help", click "Check for Updates", then click "Add" to add a new update center and enter:

Name: PMD Update Center
Location: http://pmd.sf.net/center.xml

Or download the binary release here and unzip it into your JDEV_HOME\jdev\extensions.

To use the PMD, open the Tools-&Preferences menu, click on the PMD option, and select a couple of rules to try. To run it, right click on either a Java source file or a .jpr file (i.e., a project file) and select PMD. Any rule violations should show up in a LogWindow at the bottom of the screen. You can find the viewlet for the PMD here.

For FindBugs and PMD, these two tools focus on different aspects of software quality. Tools like PMD are extremely valuable on enforcing consistent coding style guidelines, and making code easier to understand by developers. Tools like FindBugs help uncover errors, while largely ignoring style issues. Therefore, PMD and FindBugs complement each other, and neither is a good substitute for the other.

There are some comments about FindBugs and PMD, which i found out from the www.theserverside.com and give a good explanation of both as well.

"FindBugs is a very good partner with PMD because it finds issues that are "deeper" than PMD's analysis. Its explanations of the bugs it finds are excellent. The most valuable bugs it picks up for my team are missing stream closures and inconsistent synchronization behavior. We removed the EI and EI2 rules, which spend a lot of time complaining about methods that return internal arrays, which in practice is not all that evil IMHO."

"PMD is definitely the most "complete" of all the tools mentioned here. Its customizability via XSL rules or code is great, the core rulesets are broad in scope and well thought-out, it works well, and has a good Eclipse plugin. Many of the rules are fairly shallow, insofar as they are more stylistic than genuine bug-analysis rules; you also will definitely want to pick and choose which rules to use. We found it pretty easy to select a ruleset between PMD and FindBugs that we all agreed in theory having zero violations was an attainable goal. We're currently just under one failure per 100 lines of code, which is the best of any commercial project I've been on."

QJPro

QJPro is very much similar to PMD and the reason i want to mention here is because QJPro prodvide a Plugin for JDeveloper 9.0.x which PMD did not have. If developers who are using a lower version of JDeveloper like release 9.0.x and want to have a plugin like PMD, maybe QJPro can be considered. You can find the viewlet for the QJPro here.