Open Source Enterprise Service Bus & Comparison

Open Source Enterprise Service Bus in Java

Top Open Source ESB Projects

Comparison from Tijs Rademakers - Author of Open Source ESBs in Action

Mule --> Custom architecture, XML based configuration, easy for Java developers

ServiceMix 3 --> JBI based, focus on XML messages

ServiceMix 4 --> OSGi based, integrated with Camel configuration, also provides support for JBI

JBoss ESB --> Custom architecture, runs on JBoss application server, fits great with JBoss products

Synapse --> Focus on WS-*, Rest, build on Axis 2, great if you need things like WS-Security etc

OpenESB --> JBI and OSGi based, runs on Glassfish, nice tool support with Netbeans

Camel --> XML and Java DSL configuration, no container, support for EIPs and lots of transports

Spring Integration --> XML and Java annotation configuration, no container, support for EIPs

PetTALS --> JBI based, nice admin console, French based

Tuscany --> SCA based, provides support for WS-*, focus on service development not integration

THE FORRESTER ESB EVALuATIOn (Q2 2011)

The evaluation uncovered a market in which many familiar faces continue to thrive (see Figure 5):

· Software AG, Tibco, Oracle, Progress Software, and IBM are Leaders for ESB as well as CIS. These five vendors achieved Leader status in the 2009 ESB Forrester Wave evaluation and in the 2010 CIS Forrester Wave evaluation, thus garnering the top position in the integration software
provider market.

· FuseSource and WSO2 also scored as Leaders. FuseSource and WSO2 also scored highly in most of the evaluated areas; each of these vendors’ products represents a solid ESB solution that would be a good choice for meeting many enterprise integration and service-oriented architecture requirements.

· MuleSoft, IBM’s WESB, and Red Hat products scored as Strong Performers. Though MuleSoft, IBM’s WebSphere ESB (WESB), and Red Hat products were missing some features, they still made the Strong Performer category. These products lack the same level of ESB support as the Leaders, but in most cases the differences were small. Consequently, each of these products may also be a very good fit in many enterprises, depending on the specifics of the situation.

This evaluation of the enterprise service bus market is intended to be a starting point only. We
encourage readers to view detailed product evaluations and adapt the criteria weightings to fit their individual needs through the Forrester Wave Excel-based vendor comparison tool.

ESB Comparision from OpenLogic

Enterprise Service Bus Capabilities Comparison

Want to know more, read this book "Open-Source ESBs in Action"

JTS Topology Suite & GeoTools

Both are java open source tools for spatial data management.

The Open Source Java GIS Toolkit

The JTS Topology Suite is an API of 2D spatial predicates and functions. It has the following design goals:

* JTS conforms to the Simple Features Specification for SQL published by the Open GIS Consortium
* JTS provides a complete, consistent, robust implementation of fundamental 2D spatial algorithms
* JTS is fast enough for production use
* JTS is written in 100% pure JavaTM
* JTS is open source (under the LGPL license)

Find more information, you can find it from here.


Geo Tools is an open source (LGPL) Java code library which provides standards compliant methods for the manipulation of geospatial data, for example to implement Geographic Information Systems (GIS) . The Geo Tools library implements Open Geospatial Consortium (OGC) specifications as they are developed, in close collaboration with the GeoAPI and GeoWidgets projects. The capabilities of Geotools are presented in the feature list.

For more information, you can find it from here ( 1|2 ).

CVS vs SVN

For CVS, Things like tags and rolling back to older versions is far easier with CVS. And the lack of 'all or nothing' transactions is not a big problem. just add a 'build-****' tag regularly. This makes regressions very easy to track. The 'Revision Graph' in TortoiseCVS is also worth gold.

In CVS, multiple versions can get the same tag, which is logical, but with SVN it would have to 'version' the tags. A tag is a tag in CVS, but it's a folder in SVN.

Operationally speaking, maintenance of repositories and modules are easier in SVN than in CVS, in my opinion.

For development, the Tags & Branches function is very important which CVS is better than SVN. As SVN is actually using different approaches for tagging ( Basically, a tag and a branch, for that matter in SVN is kind of like a copy. However, it's more of a "smart" copy).

CVS Convention and Best Practice
http://tldp.org/REF/CVS-BestPractices/html/index.html

How to install CVS in RedHat Enterprise
http://personal.vsnl.com/sureshms/linuxindex.html
http://www.michael-amorose.com/articles/computers/cvs/6_1.html
http://www.taursys.com/howto/cvs/
http://www.faqs.org/docs/ldev/0130091154_198.htm

CVS Installation for Windows
http://www.devguy.com/fp/cfgmgmt/cvs/cvs_admin_nt.htm

useradd -r -c "CVS user" cvs
passwd cvs
cvs -d /usr/local/cvsroot init
chown -R cvs.cvs /usr/local/cvsroot
cvs -d :pserver:daps@localhost:/usr/local/cvsroot login

CVS Tutorial Link
http://www.cvshome.org/
http://www.developingprogrammers.com/index.php/2005/11/24/cvs-and-subversion-combined-tutorial/

CVS vs SVN reference Link

http://www.pushok.com/soft_svn_vscvs.php
http://www.devmaster.net/forums/showthread.php?t=7394
http://svn.haxx.se/users/archive-2006-10/1382.shtml
http://lists.danga.com/pipermail/mogilefs/2006-June/000336.html
http://wiki.scummvm.org/index.php/CVS_vs_SVN
http://www.gigascale.org/softdevel/faq/27.html
http://users.footprints.net/~kaz/mcvs-svn-comparison.html

SOA, Web Services & BPEL - Tech Forum in Singapore

Recently, i gave a talk and BPEL demo to singapore partners for Oracle BPEL & SOA suites. Introducing them idea of SOA, BPEL and Web Services and how to make use of oracle technology in their applications.

SOA stands for service oriented architecture, it is a architecture model.

Talk about architecture, we need to understand basic building block of each application and system.

Like we build a real house, we use mortar and brick. Every application / system was built with business logic and process. Business process weaves all business logics into one application to achieve business goals. Process and logic weave seamlessly and hard coded into a monolithic application. Like we cook soup, we put all ingredients into water and hard for us to separate each out in the end, by which it reduce reusability of standard process and logics.

In traditional application, when people wanna change the process or integrated with other application / system, especially it is a heterogeneous platform or technology integration, that will be very hard and time-consuming.

For SOA applications, applications also build with process and so-called business logic (have been wrapped into self-contained services). Process and services are loosely coupled, people can easily add or remove services from process, not like traditional applications (process & logic fuse into one). Other than that, people can easily design & program the business process with a advanced IDE. Overall experience for process design and implementation will be a drag-and-drop exercise.

SOA process should be standard, easily extensible and composable. When build an SOA application, we start from building a portfolio of services , follow with well-defined business process to orchestrate services to achieve business goal. This is where BPEL comes in for business process implementation and web service for business service implementation.

Many people equate web services with Service-Oriented Architectures (SOA), but web services are just one element of SOA - they're about service invocation. SOA's scope is broader than just invocation. It's a complete software component architectural model, a way to organize, manage, deploy, discover, and expose software components, and to consume these over the Internet. From the software life-cycle perspective, in addition to the typical software development phases of design, development, and deployment, SOA defines web services-specific phases that include service publishing, discovery, binding and consumption, and management.

Note, SOA != web service+bpel

ESB vs BPEL

Both ESB and BPEL can be used for SOA implementation.

ESB (Enterprise Service Bus) implement messaging to enable services to be integrated in a message-based paradigm: both synchronous and asynchronous styles. ESB moves data via different adaptors (WebServices, FTP,File,JDBC etc) and protocols (HTTP,JMS) and enriches and transform data using XSL & domain value mapping lookups.

Oracle BPEL provides a comprehensive, standards-based and easy to use solution for creating, deploying, and managing cross-application business processes with both automated and human workflow steps.

ESB is good for routing messages to multiple destinations. It is also good for doing transformations that have little to no business rules. The footprint is much smaller and incurs minimal overhead therefore the performance is much better.

BPEL is used for bringing together multiple services. There is much more functionality and allows implementation of complex business logic.

• BPEL data transformations enrich and perform complex changes whereas ESB perform only simple transformations.
• BPEL can use Business rules, Human Workflow and Notifications, exception handling can be done in BPEL.

Use ESB when you want a really low cost solution, Only need connectivity, simple transformations and routing.

BPEL is primarily used for servies orchestration, data enrichment and human interaction.

Use BPEL+ESB when, You need to code complex business logic, integration with Workflow, complex transforms potentially involving the database or having long running processes which may or may not be stateful.

With an appropriately layered SOA architecture, one would look at ESB as a repository of services that BPEL orchestrates. At deployment time, this layered approach provides you the flexibility to dedicate servers to individual tiers (BPEL and ESB) and thereby maximize the resource usage for your particular use case.

Oracle TAF & FCF

Transparent Application Failover (TAF)

Transparent Application Failover (TAF) is a feature of the Java Database Connectivity (JDBC) Oracle Call Interface (OCI) driver. It enables the application to automatically reconnect to a database, if the database instance to which the connection is made fails. In this case, the active transactions roll back.

Transparent Application Failover (TAF) is a client-side feature that allows for clients to reconnect to surviving databases in the event of a failure of a database instance. Notifications are used by the server to trigger TAF callbacks on the client-side.

TAF is configured using either client-side specified TNS connect string or using server-side service attributes. However, if both methods are used to configure TAF, the server-side service attributes will supersede the client-side settings. The server-side service attributes are the preferred way to set up TAF.

TAF can operate in one of two modes, Session Failover and Select Failover. Session Failover will recreate lost connections and sessions. Select Failover will replay queries that were in progress.

Transparent Application Failover
Configuring Transparent Application Failover

Fast Connection Failover (FCF)

Fast Connection Failover offers a driver-independent way for your Java Database Connectivity (JDBC) application to take advantage of the connection failover facilities offered by Oracle Database 10g. The Fast Connection Failover mechanism depends on the implicit connection cache feature. As a result, for Fast Connection Failover to be available, implicit connection caching must be enabled.

FCF vs TAF

TAF is always active and does not have to be set, it does not work with the OCI Connection Pool.
Oracle recommends not to use TAF and Fast Connection Failover in the same application

XML-RPC vs JAX-RPC

XML-RPC

XML-RPC is a remote procedure call protocol which uses XML to encode its calls and HTTP as a transport mechanism. XML-RPC is a very simple protocol, defining only a handful of data types and commands, and the entire description can be printed on two pages of paper. Some people still prefer XML-RPC to SOAP because of its simplicity, minimalism, and ease of use. (source wikipedia)

The original XML-RPC uses only a few data types and does not require namespaces. It achieves considerable flexibility by defining and types which can contain other values. Responses are also simple XML with as the root element with either or as the child element. The contrast with complex SOAP responses is striking.

For xml-rpc, wsdl and uddi is not involved for client to invoke the web service.

Implementations of XML-RPC:

There are five Implementations in Java list at xmlrpc.com as of December 2003. However, many of these projects appear to be inactive. The project hosted by the Apache Web Services Project is active. It looks like the Apache Web Services Project is on the right track with version 3, so keep an eye open on it if you are interested in an open-source XML-RPC solution.

JAX-RPC - The Java API for XML-based RPC

Instead of using xml-rpc protocol, JAX-RPC make used of SOAP protocol for messaging which enable web service a industry standard.

SOAP

SOAP is a protocol for exchanging XML-based messages over computer networks,normally using HTTP/HTTPS. SOAP forms the foundation layer of the Web services stack, providing a basic messaging framework so that more abstract layers can build on.

There are several different types of messaging patterns in SOAP, but by far the most common is the Remote Procedure Call (RPC) pattern, in which one network node (the client) sends a request message to another node (the server) and the server immediately sends a response message to the client. SOAP is the successor of XML-RPC, though it borrows its transport and interaction neutrality and the envelope/header/body from elsewhere, probably from WDDX. (Source: wikipedia)

WSDL is involved for client to invoke the web service when use JAX-RPC.

Is XML-RPC Still Useful?

The complexity of the JAX-RPC implementation has caused somewhat of a backlash among programmers. It seems you should consider XML-RPC over JAX-RPC if:

* Your application uses relatively simple requests
* You control both server and client sides and/or you don't need to publish a WSDL
* You want to keep client-side parsing simple (as in AJAX applications)

JAXM & Messaging

JAXM Definition

JAXM, the Java API for XML Messaging, defines a lightweight Java API for composing, processing, sending, and receiving XML documents. It supports both synchronous (HTTP) and asynchronous (SMTP) messaging and “Out-of-the-box” support for ebXML messaging.

Fundamental Elements

The JAXM Service

A JAXM service consumes JAXM messages sent by a JAXM client. To develop a JAXM service, developers extend the class javax.xml.messaging.JAXMServlet and implement either the javax.xml.messaging.OnewayListener or javax.xml.messaging.ReqRespListener interface. The choice of interface depends upon whether the interaction between the client and the server is asynchronous (OnewayListener) or synchronous (ReqRespListener) in nature.

The JAXM Client

Clients exchange messages with JAXM services. JAXM clients can either interact directly with a JAXM service or go through a JAXM provider. JAXM clients that interact directly with a JAXM service can only participate in synchronous interactions. JAXM clients that use a messaging provider can participate in asynchronous as well as synchronous interactions with the JAXM service.

The JAXM Messaging Provider

A JAXM messaging provider is responsible for managing the routing of messages for a JAXM client. Besides providing a degree of separation between a JAXM client and service, a provider can also offer additional services like reliable messaging, security, and transaction support. Currently, clients that want to interact asynchronously with a JAXM service have to use a provider.

The JAXM Message

All JAXM messages conform to the SOAP 1.1 and SOAP with Attachments standards. JAXM messages also support the concept of higher-level protocols like ebXML through the concept of messaging profiles. A messaging profile defines the messaging contract between two parties. A JAXM client and service that share a message profile like ebXML can conduct business by exchanging ebXML messages.

JAXM & Messaging

* JAXM is an synchronous / asynchronous (queue-based) messaging API
* Not part of J2EE 1.4
* Uses queues on either side
* Doesn’t provide WSDL facilities
* Can be used in conjunction with JAXB to convert to objects

Difference in JAXM & JMS

The JAXM API was designed for Internet SOAP-based messaging, while the JMS API was designed as a common messaging API to be layered over existing enterprise message-oriented middleware (MOM) applications.

The delivery endpoint models differ. JAXM provides direct point-to-point messaging, while JMS uses destinations to provide a reliable asynchronous messaging capability.

JMS clients send messages to, and retrieve them from, destinations. JMS provides a model whereby an intermediary, a messaging server, makes sure a message is delivered between two applications that do not know directly about each other, but know only about the common destination.

A JAXM client is interoperable over any SOAP 1.1 compatible client (there must be an agreed-upon profile between providers, however), whereas a JMS client is interoperable only with other JMS clients over the same messaging system (or another messaging system for which the JMS provider supports a proprietary gateway).

The JMS API does not define interoperability between message providers. It does not define a common message format, only a number of message types; the clients must agree on the precise format. (It is possible to wrap a SOAP message, or any other kind of XML message, in a TextMessage and send it via JMS.)

Find out more info from JAXM FAQ

Why there is a need for JAXM?

The goal of JAXM is to provide a rich set of interfaces for document-style web services. Document-style web services enable the exchange of XML documents between two parties, as opposed to RPC-style web services, which expose software functions as web services.

JAXM-based web services can be used effectively in Such scenario:

When two parties want to exchange data using XML documents that are bounded using well-defined XML schemas instead of invoking software functions (Java objects, C procedures, etc.) exposed as RPC-style web services.

I would like to recommend you to go through a article called "Developing E-Business Interactions with JAXM" for a good understanding of How & Where to use JAXM

Implementation

Sun provides JAXM implementation, but does not support full ebXML messaging service specification, there is alternative implementation of ebMS 2.0 from www.freebxml.org.

ebXML & Message Service

Find more from my previous post here.
Find more from article called "ebXML Message Service - An In-depth Overview"

Reference List:

http://java.sun.com/webservices/jaxm
Sun JAXM tutorial
Developing E-Business Interactions with JAXM

JAX-RPC vs JAX-WS

Java API for XML-Based RPC (JAX-RPC) is a Legacy Web Services Java API, it uses SOAP and HTTP to do RPCs over the network and enables building of Web services and Web applications based on the SOAP 1.1 specification, Java SE 1.4 or lower, or when rpc/encoded style must be used.

You can use the JAX-RPC programming model to develop SOAP-based web service clients and endpoints. JAX-RPC enables clients to invoke web services developed across heterogeneous platforms. Likewise, JAX-RPC web service endpoints can be invoked by heterogeneous clients. JAX-RPC requires SOAP and WSDL standards for this cross-platform interoperability.

JAX-RPC lets people develop a web service endpoint using either a Servlet or Enterprise JavaBeans (EJB) component model. The endpoint is then deployed on either the Web or EJB container, based on the corresponding component model. Endpoints are described using a Web Services Description Language (WSDL) document.(This WSDL document can be published in a public or private registry, though this is not required). A client then uses this WSDL document and invokes the web service endpoint.

JAX-RPC in J2ee 1.4 supports 4 types of stubs and invocations: static stub, dynamic proxy, Dynamic Invocation Interface (DII) and Application client.

Static Stub Client
Web service client makes a call through a stub, a local object that acts as a proxy for the remote service. Because the stub is created by wscompile at development time (as opposed to runtime), it is usually called a static stub.

Example: Invoking a Stub Client
Stub stub = (Stub) (new MyHelloService_Impl().getHelloIFPort());
stub._setProperty (javax.xml.rpc.Stub.ENDPOINT_ADDRESS_PROPERTY,endpoint_address_string);
HelloIF hello = (HelloIF)stub;
System.out.println(hello.sayHello("Duke!"));

Dynamic Proxy Client
In contrast, the client call a remote procedure through a dynamic proxy, a class that is created during runtime. Although the source code for the static stub client relies on an implementation-specific class, the code for the dynamic proxy client does not have this limitation.

Example: Dynamic Proxy
javax.xml.rpc.Service service = ServiceFactory.newInstance().createService(...);
com.example.StockQuoteProvider sqp = (com.example.StockQuoteProvider)service.getPort(portName, StockQuoteProvider.class);
float price = sqp.getLastTradePrice("ACME");

Dynamic Invocation Interface Client
With the dynamic invocation interface (DII), a client can call a remote procedure even if the signature of the remote procedure or the name of the service is unknown until runtime. In contrast to a static stub or dynamic proxy client, a DII client does not require runtime classes generated by wscompile.

Example: Dynamic Invocation Interface
javax.xml.rpc.Service service = ServiceFactory.newInstance().createService(...);
javax.xml.rpc.Call call = service.createCall(portName, "getLastTradePrice");
// This example assumes that addParameter and setReturnType methods are not required to be called
Object[] inParams = new Object[] {"ACME"};
Float quotePrice = (Float)call.invoke(inParams);

Application Client
Unlike the stand-alone clients, for an application client, because it's a J2EE component, an application client can locate a local web service by invoking the JNDI lookup method.

Example: Application Client
Context ic = new InitialContext();
MyHelloService myHelloService = (MyHelloService)
ic.lookup("java:comp/env/service/MyJAXRPCHello");
appclient.HelloIF helloPort = myHelloService.getHelloIFPort();
((Stub)helloPort)._setProperty(Stub.ENDPOINT_ADDRESS_PROPERTY,args[0]);
System.out.println(helloPort.sayHello("Jake!"));

Service Endpoint Model

JAX-RPC supports a client model for the service consumer, and a service endpoint model for the service producer.

Application-Level Interaction Modes

JAX-RPC specifies three client application interaction models:
* Synchronous request-response two-way RPC
* Asynchronous (non-blocking) request-response two-way RPC
* One-way RPC

JAX-WS 2.0 is the successor of JAX-RPC 1.1 - the Java API for XML-based Web services. If possible, JAX-WS should be used instead as it is based on the most recent industry standards.

What remains the same?

Before we itemize the differences between JAX-RPC 1.1 and JAX-WS 2.0, we should first discuss what is the same.

* JAX-WS still supports SOAP 1.1 over HTTP 1.1, so interoperability will not be affected. The same messages can still flow across the wire.

* JAX-WS still supports WSDL 1.1, so what you've learned about that specification is still useful. A WSDL 2.0 specification is nearing completion, but it was still in the works at the time that JAX-WS 2.0 was finalized.

What is different?

* SOAP 1.2
JAX-RPC and JAX-WS support SOAP 1.1. JAX-WS also supports SOAP 1.2.

* XML/HTTP
The WSDL 1.1 specification defined an HTTP binding, which is a means by which you can send XML messages over HTTP without SOAP. JAX-RPC ignored the HTTP binding. JAX-WS adds support for it.

* WS-I's Basic Profiles
JAX-RPC supports WS-I's Basic Profile (BP) version 1.0. JAX-WS supports BP 1.1. (WS-I is the Web services interoperability organization.)

* New Java features
o JAX-RPC maps to Java 1.4. JAX-WS maps to Java 5.0. JAX-WS relies on many of the features new in Java 5.0.
o Java EE 5, the successor to J2EE 1.4, adds support for JAX-WS, but it also retains support for JAX-RPC, which could be confusing to today's Web services novices.

* The data mapping model
o JAX-RPC has its own data mapping model, which covers about 90 percent of all schema types. Those that it does not cover are mapped to javax.xml.soap.SOAPElement.
o JAX-WS's data mapping model is JAXB. JAXB promises mappings for all XML schemas.

* The interface mapping model
JAX-WS's basic interface mapping model is not extensively different from JAX-RPC's; however:
o JAX-WS's model makes use of new Java 5.0 features.
o JAX-WS's model introduces asynchronous functionality.

* The dynamic programming model
o JAX-WS's dynamic client model is quite different from JAX-RPC's. Many of the changes acknowledge industry needs:
+ It introduces message-oriented functionality.
+ It introduces dynamic asynchronous functionality.
o JAX-WS also adds a dynamic server model, which JAX-RPC does not have.

* MTOM (Message Transmission Optimization Mechanism)
JAX-WS, via JAXB, adds support for MTOM, the new attachment specification. Microsoft never bought into the SOAP with Attachments specification; but it appears that everyone supports MTOM, so attachment interoperability should become a reality.

* The handler model
o The handler model has changed quite a bit from JAX-RPC to JAX-WS.
o JAX-RPC handlers rely on SAAJ 1.2. JAX-WS handlers rely on the new SAAJ 1.3 specification.

Additional reading list:

Developing Web Services at a Glance
http://java.sun.com/webservices/technologies/index.jsp#Core_Web_Services
Java API for XML-based RPC (JAX-RPC)
http://java.sun.com/developer/technicalArticles/xml/jaxrpc/

Reference:
http://java.sun.com/webservices/technologies/index.jsp#Core_Web_Services
http://www-128.ibm.com/developerworks/webservices/library/ws-tip-jaxwsrpc.html
http://www.ibm.com/developerworks/webservices/library/ws-tip-jaxwsrpc2.html

Keywords: SAAJ, JAXB, SOAP, JAX-RPC, JAX-WS

Vector graphics Vs Raster graphics

Source: wikimedia.org
Vector graphics (also called geometric modeling or object-oriented graphics) is the use of geometrical primitives such as points, lines, curves, and polygons, which are all based upon mathematical equations to represent images in computer graphics.

It is used in contrast to the term raster graphics, which is the representation of images as a collection of pixels, and used as the sole graphic type for actual photographic images. To know more about vector | raster

Oracle Locator Vs Oracle Spatial

To get a quick idea of what difference between oracle locator and spatial are, i put them down here.

Features included in Oracle Locator:

* Exactly the same code base as Oracle Spatial for included functions - you won't get a performance degrade by using Oracle Locator, and features behave in the same manner in Oracle Locator and Oracle Spatial;
* Full support to store spatial data types;
* Standard SQL query access to spatial data;
* All indexing types are available - including R-Tree and partition support;
* No limits on the size and complexity of data stored;
* Most of the common spatial operations, such as "find all customers in this region" and "what shops are within 100 metres of this school", are available;
* Data compatibility between Oracle Spatial and Locator, so if you do decide to upgrade, all your data remains valid; and
* Support for multiple coordinate systems.

Oracle Spatial provides the following functions which are not included in Oracle Locator:

* Linear referenced features e.g. roads which store the distance along the road;
* Network models, such as road networks, allowing rapid calculation of shortest paths;
* Topological relationships can be stored, such as this property boundary touches this neighbouring boundary. Generally used by data producers wishing to edit high integrity spatial data;
* Storage of images such as satellite and aerial photography;
* A geocoder that translates address data to coordinate; and
* Some more complex spatial functions, mainly for analysis and data aggregation.

Find the original article here.

There a list of relevant readings from where you can go further.

Oracle Spatial - Wikipedia
Oracle Spatial Documents - Documentation
Oracle Locator and Spatial Option - Datasheet
Oracle Locator and Spatial FAQ - FAQ
Oracle XE , Locator & Geospatial Information Systems - Interview

How to select Code Analysis Tools

Code checking tools are software products that check compile-time whether a software program adheres to a certain set of rules of a coding standard. Without such code checkers, the enforcement of a coding standard in an organization is likely to fail. There are two main causes for this: (1) the number of rules in a coding standard is usually so large that nobody can remind them all (except the author) and (2) some context-sensitive rules that demand inspection of several files are very hard to check by human beings.

Although code checkers can reduce formal code review time considerably, it certainly does not mean that code reviews are not needed anymore. Some coding rules cannot be checked at all (e.g. "Comments should be in English") or only partly at compile-time (e.g. "Use delete, whenever you use new"). On average about 65% of a coding standard can be covered with code checking tools.

There are several characteristics that should be assessed during the selection process of a code checker. There are 6 issues that are most important. Obvious requirements such as whether it is available for the used platform are not taken into account.

Availability to define own rules. Experience has shown that only about 20% of a propietary coding standard is available as built-in rule in a code checker. As a consequence, there should be a possibility to define own rules. This can be via an API (complex in use but powerful) or a graphical user interface (easy to use but limited).

Integration in programming IDE. Most end users of code checkers are software engineers. Therefore a code checker should fit smoothly in the used programming environment. Preferrably via a toolbar with an easy way of running the tool (one button push).

Presence of command-line version. In order to be able to integrate the code checker in the software development process and/or in the programming IDE (if no plug-in is available) it should be command-line oriented. Command-line versions also allow for nightly batch runs of the code checker to collect quality data.

High performance. Code checkers are slower than compilers by nature because they perform a more in-depth semantical analysis of the source code. However, performance is a key issue for end user acceptance of the tooling. Response times of more than 1 minute per file are perceived as not workable.

Low cost of ownership. Cost of ownership means purchase price and maintenance costs. Since code checkers are not used continuously by end users, floating licenses appear to be more cost effective in most cases. About 1 floating end user license is needed per 10 developers.

Available support. Decent product support is an essential feature of a code checker. Good support does not only mean prompt and adequate response to user questions and problems, but also the frequency of new product release. "We will solve this in the next release" is not interesting if the average product release cycle is 1 year.

This How-to is an excerpt from a article called "Industrial methodology to introduce coding standards" and i feel the considerations and points are logical and good to follow for the code analyzers evaluation.

Evaluation of three open source code analysers

There are couple of code analysis tools avaiable out there and three of them (FindBugs, PMD and QJPro) i have evaluated and would like to elaborate some more. I will give a quick demo for each along the way and it will give you a good visualization of how each tool looks like in the real life.

FindBugs

FindBugs looks for bugs in java the programs. It can detect a variety of common coding mistakes, including thread synchronization problems, misuse of API methods and etc.

FindBugs is a static analysis tool that examines java classes or JAR files and looks for potential problems by matching class bytecodes against a list of bug patterns. With static analysis tools, we can analyze software without actually running the program. FindBugs requires JRE (or JDK) 1.4.0 or later to run. However, it can analyze programs compiled for any version of Java. FindBugs supports a plugin architecture allowing anyone to add new bug detectors, which means we can write custom detectors to find application-specific problems and it has been used for Eclipse and Java JDK project for bugs finding.

We can run FindBugs in multiple ways -- from a GUI, from a command line, using Ant, as an Eclipse plug-in, and using Maven. However, it makes more sense to run it as an integrated part of nightly build infrastructure and generate a bug report, which could be easily done by using tools like CruiseControl, Anthill, or even simple cron to fire off nightly builds and giving the results as an email. FindBugs certainly won't find all the bugs, but they'll help find some of them. FindBugs can act as a safety net and detects identified bug patterns when developers check in their codes into the cvs. You can find a demo for FindBugs here and a viewlet here.

PMD

PMD is not like FindBugs and it analyzes the java source code instead of class files, scans and looks for potential problems like:

* Possible bugs - empty try/catch/finally/switch statements
* Dead code - unused local variables, parameters and private methods
* Suboptimal code - wasteful String/StringBuffer usage
* Overcomplicated expressions - unnecessary if statements, for loops that could be while loops
* Duplicate code - copied/pasted code means copied/pasted bugs


PMD has over 175 “rules”, which are undesirable patterns of code, right out of the box. These rules are grouped into “rulesets,” such as Basic, Unused Code Design, Optimization, Strict Exception, String and StringBuffer, Naming, JUnit and Java Logging.

Not all of them will be appropriate for our project or organization. Fortunately, PMD is configurable and we can configure which rules we want enforced. To do this, we can make our own ruleset file, which states which of the rules we want executed during analysis.

PMD is integrated with Eclipse, JBuilder, IntelliJ IDEA, Maven, Ant, and JDeveoper release 10.1.2 and 10.1.3.

It can also be run on demand from the command line, which allows it to be integrated into nightly Ant/Maven builds and publish the results to a shared location such as an intranet or file server.

There are two ways we can install the PMD plugin for JDeveloper release 10.1.3. One way is to install the extension by using JDeveloper's built in "update center" functionality. Fire up JDeveloper, click "Help", click "Check for Updates", then click "Add" to add a new update center and enter:

Name: PMD Update Center
Location: http://pmd.sf.net/center.xml

Or download the binary release here and unzip it into your JDEV_HOME\jdev\extensions.

To use the PMD, open the Tools-&Preferences menu, click on the PMD option, and select a couple of rules to try. To run it, right click on either a Java source file or a .jpr file (i.e., a project file) and select PMD. Any rule violations should show up in a LogWindow at the bottom of the screen. You can find the viewlet for the PMD here.

For FindBugs and PMD, these two tools focus on different aspects of software quality. Tools like PMD are extremely valuable on enforcing consistent coding style guidelines, and making code easier to understand by developers. Tools like FindBugs help uncover errors, while largely ignoring style issues. Therefore, PMD and FindBugs complement each other, and neither is a good substitute for the other.

There are some comments about FindBugs and PMD, which i found out from the www.theserverside.com and give a good explanation of both as well.

"FindBugs is a very good partner with PMD because it finds issues that are "deeper" than PMD's analysis. Its explanations of the bugs it finds are excellent. The most valuable bugs it picks up for my team are missing stream closures and inconsistent synchronization behavior. We removed the EI and EI2 rules, which spend a lot of time complaining about methods that return internal arrays, which in practice is not all that evil IMHO."

"PMD is definitely the most "complete" of all the tools mentioned here. Its customizability via XSL rules or code is great, the core rulesets are broad in scope and well thought-out, it works well, and has a good Eclipse plugin. Many of the rules are fairly shallow, insofar as they are more stylistic than genuine bug-analysis rules; you also will definitely want to pick and choose which rules to use. We found it pretty easy to select a ruleset between PMD and FindBugs that we all agreed in theory having zero violations was an attainable goal. We're currently just under one failure per 100 lines of code, which is the best of any commercial project I've been on."

QJPro

QJPro is very much similar to PMD and the reason i want to mention here is because QJPro prodvide a Plugin for JDeveloper 9.0.x which PMD did not have. If developers who are using a lower version of JDeveloper like release 9.0.x and want to have a plugin like PMD, maybe QJPro can be considered. You can find the viewlet for the QJPro here.