Tuesday, November 20, 2007

Why Wiki , Blogging & Social Computing

Blogging and wiki is something about the knowledge management, one step further than the information management. There are many ways of knowledge management, blogging and wiki are just one of them.

why we bother to blog or wiki it rather than make a phone call, send a email or book a meeting room? Like it or not, social computing is part of how organization works today and an even larger part of how it will work tomorrow.

Social computing is what happens when people selectively consume, produce, and share online content in new ways. Technologies like blogs, peer-to-peer networking, RSS, wikis, and other collaboration tools continue to grow, evolve, and feed the demand for new methods of interaction. The challenges that you face today as a result of access to online information, software, and communities will intensify, with new challenges emerging over time. To maintain influence in this new world, you must embrace social computing, not fight it. You must become a facilitator for this kind of collaboration and a full member of the social computing community.

Develop your interest in using these forces to do your job more effectively, rather than letting them just make your job more difficult. Instead of seeing this as a new onslaught of back-door paths for unwanted change to reach your organization, adopt a leadership role in using these tools to create a more collaborative and productive organization.

Don't fight this trend, embrace it. You should become an advocate and facilitator for social computing. Make sure people can get what they need within the organization so that they won't have to go elsewhere. Make yourself part of the social computing community, and use that insider advantage to help the organization succeed.

As we take advantage of social computing, don't neglect traditional means of communicating and promoting collaboration. Face-to-face idea sharing remains an important tool in building and nurturing critical relationships, and we should continue to use it for communication.

KM Tool - Wiki

Wikis are tools that allow for collaborative editing and publishing and hence the creation of user generated content. Wikis are another example of social software - software that encourages people to work together.

Wikis are a useful tool when you want people to collaborate and work together on creating a common document, e.g. a project document, a manual, and so on or to work on a common strategy or event, e.g. a conference programme.

In corporate and general use they can be used to create collaborative documents, manuals e.g. WikiHow (a collaborative Manual)

The most obvious and famous example of a wiki is of course Wikipedia, which is an encyclopaedia that has been created by thousands of individuals. Every page in Wikipedia can be edited, which means it is constantly evolving and growing and can be kept up to date.

In the commercial tools market there are a number of enterprise wiki tools available, for example, e.g. Confluence

But there are also a number of free tools, for example:

* MediaWiki is the software used for Wikipedia and the other Wikimedia Foundation websites. Compared to other wikis, it has an excellent range of features and support for high-traffic websites using multiple servers (Wikipedia peaks in the 2500+ requests per second range as of June 2005).
* pbWiki is another hosted service and allows you to set up a wiki very quickly, as are Wikispaces and Wetpaint
* If you are looking for a personal, notepad-type wiki to download onto your own computer, then TiddlyWiki is a useful tool

Reference list

Mediawiki Installation & File Upload HowTO
http://www.washington.edu/computing/web/publishing/mediawiki.html

MediaWiki Comparison Matrix
http://www.wikimatrix.org/show/MediaWiki

How to install PHP and MySQL
http://www.ozzu.com/ftopic56303.html

Wikis - and how to create them quickly and easily

http://c4lpt.co.uk/handbook/wikis.html

Wednesday, November 14, 2007

CVS vs SVN

For CVS, Things like tags and rolling back to older versions is far easier with CVS. And the lack of 'all or nothing' transactions is not a big problem. just add a 'build-****' tag regularly. This makes regressions very easy to track. The 'Revision Graph' in TortoiseCVS is also worth gold.

In CVS, multiple versions can get the same tag, which is logical, but with SVN it would have to 'version' the tags. A tag is a tag in CVS, but it's a folder in SVN.

Operationally speaking, maintenance of repositories and modules are easier in SVN than in CVS, in my opinion.

For development, the Tags & Branches function is very important which CVS is better than SVN. As SVN is actually using different approaches for tagging ( Basically, a tag and a branch, for that matter in SVN is kind of like a copy. However, it's more of a "smart" copy).

CVS Convention and Best Practice
http://tldp.org/REF/CVS-BestPractices/html/index.html

How to install CVS in RedHat Enterprise
http://personal.vsnl.com/sureshms/linuxindex.html
http://www.michael-amorose.com/articles/computers/cvs/6_1.html
http://www.taursys.com/howto/cvs/
http://www.faqs.org/docs/ldev/0130091154_198.htm

CVS Installation for Windows
http://www.devguy.com/fp/cfgmgmt/cvs/cvs_admin_nt.htm

useradd -r -c "CVS user" cvs
passwd cvs
cvs -d /usr/local/cvsroot init
chown -R cvs.cvs /usr/local/cvsroot
cvs -d :pserver:daps@localhost:/usr/local/cvsroot login

CVS Tutorial Link
http://www.cvshome.org/
http://www.developingprogrammers.com/index.php/2005/11/24/cvs-and-subversion-combined-tutorial/

CVS vs SVN reference Link

http://www.pushok.com/soft_svn_vscvs.php
http://www.devmaster.net/forums/showthread.php?t=7394
http://svn.haxx.se/users/archive-2006-10/1382.shtml
http://lists.danga.com/pipermail/mogilefs/2006-June/000336.html
http://wiki.scummvm.org/index.php/CVS_vs_SVN
http://www.gigascale.org/softdevel/faq/27.html
http://users.footprints.net/~kaz/mcvs-svn-comparison.html

Monday, November 5, 2007

SOA, Web Services & BPEL - Tech Forum in Singapore

Recently, i gave a talk and BPEL demo to singapore partners for Oracle BPEL & SOA suites. Introducing them idea of SOA, BPEL and Web Services and how to make use of oracle technology in their applications.

SOA stands for service oriented architecture, it is a architecture model.

Talk about architecture, we need to understand basic building block of each application and system.

Like we build a real house, we use mortar and brick. Every application / system was built with business logic and process. Business process weaves all business logics into one application to achieve business goals. Process and logic weave seamlessly and hard coded into a monolithic application. Like we cook soup, we put all ingredients into water and hard for us to separate each out in the end, by which it reduce reusability of standard process and logics.

In traditional application, when people wanna change the process or integrated with other application / system, especially it is a heterogeneous platform or technology integration, that will be very hard and time-consuming.

For SOA applications, applications also build with process and so-called business logic (have been wrapped into self-contained services). Process and services are loosely coupled, people can easily add or remove services from process, not like traditional applications (process & logic fuse into one). Other than that, people can easily design & program the business process with a advanced IDE. Overall experience for process design and implementation will be a drag-and-drop exercise.

SOA process should be standard, easily extensible and composable. When build an SOA application, we start from building a portfolio of services , follow with well-defined business process to orchestrate services to achieve business goal. This is where BPEL comes in for business process implementation and web service for business service implementation.

Many people equate web services with Service-Oriented Architectures (SOA), but web services are just one element of SOA - they're about service invocation. SOA's scope is broader than just invocation. It's a complete software component architectural model, a way to organize, manage, deploy, discover, and expose software components, and to consume these over the Internet. From the software life-cycle perspective, in addition to the typical software development phases of design, development, and deployment, SOA defines web services-specific phases that include service publishing, discovery, binding and consumption, and management.

Note, SOA != web service+bpel

ESB vs BPEL

Both ESB and BPEL can be used for SOA implementation.

ESB (Enterprise Service Bus) implement messaging to enable services to be integrated in a message-based paradigm: both synchronous and asynchronous styles. ESB moves data via different adaptors (WebServices, FTP,File,JDBC etc) and protocols (HTTP,JMS) and enriches and transform data using XSL & domain value mapping lookups.

Oracle BPEL provides a comprehensive, standards-based and easy to use solution for creating, deploying, and managing cross-application business processes with both automated and human workflow steps.

ESB is good for routing messages to multiple destinations. It is also good for doing transformations that have little to no business rules. The footprint is much smaller and incurs minimal overhead therefore the performance is much better.

BPEL is used for bringing together multiple services. There is much more functionality and allows implementation of complex business logic.

• BPEL data transformations enrich and perform complex changes whereas ESB perform only simple transformations.
BPEL can use Business rules, Human Workflow and Notifications, exception handling can be done in BPEL.

Use ESB when you want a really low cost solution, Only need connectivity, simple transformations and routing.

BPEL is primarily used for servies orchestration, data enrichment and human interaction.


Use BPEL+ESB when, You need to code complex business logic, integration with Workflow, complex transforms potentially involving the database or having long running processes which may or may not be stateful.

With an appropriately layered SOA architecture, one would look at ESB as a repository of services that BPEL orchestrates. At deployment time, this layered approach provides you the flexibility to dedicate servers to individual tiers (BPEL and ESB) and thereby maximize the resource usage for your particular use case.

Friday, November 2, 2007

Io exception: The Network Adapter could not establish the connection

Recently, i did an RAC validation with our oracle partner and encountered this weird exception called "Io exception: The Network Adapter could not establish the connection".

Two client machine were running same java code (using jdbc thin driver & oracle datasource) in the same LAN as RAC servers, but they came out with differen behaviour. One could connect to RAC database, the other one always got io exception mentioned above.

The only difference between two machines was one of clients' host file having rac ip & hostname mapping entry, which is able to connect to oracle rac db.

Something weird is that all db clients and servers were actually sitting in the same LAN. I could ping and telnet db rac with rac ip address (virtual and real ) from client machines, but when i use ip address in the java class to lookup the db connection, i could not get connected to db if rac ip & hostname mapping entries were not available in the client host file.

Here is the sample Codes for RAC FCF connection:

OracleDataSource ods = new OracleDataSource();
OracleConnectionCacheManager occm = null;
ods.setUser("user-name");
ods.setPassword("password");
String cacheName = "cache";
String dbURL="jdbc:oracle:thin:@"+
"(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=ON)" +
"(ADDRESS=(PROTOCOL=TCP)" +
"(HOST=rac1-vip-address)(PORT=1521))" +
"(ADDRESS=(PROTOCOL=TCP)" +
"(HOST=rac2-vip-address)(PORT=1521)))" +
"(CONNECT_DATA=(SERVICE_NAME=orcl.abc.com)))";
ods.setURL(dbURL); System.out.println("Url=" + dbURL ); ods.setConnectionCachingEnabled(true);
Properties prop = new Properties();
prop.setProperty("MinLimit", "5");
prop.setProperty("MaxLimit", "40");
prop.setProperty("InitialLimit", "10");
prop.put (oracle.net.ns.SQLnetDef.TCP_CONNTIMEOUT_STR,"" + (5000)); // 5000ms ods.setConnectionCacheProperties(prop);
ods.setFastConnectionFailoverEnabled(true);
ods.setONSConfiguration("nodes=rac1-ip-address:6200,rac2-ip-address:6200"); ods.setConnectionCacheName(cacheName);
//occm = OracleConnectionCacheManager.getConnectionCacheManagerInstance();
Connection conn = ods.getConnection();

It looks somewhere JDBC driver looking up db connection with rac hostname instead of ip address and I have to put the rac ip & hostname mapping in the client host file for jdbc driver to resolve them.

Wednesday, October 31, 2007

Web service security & clustering

The Oracle Web Services Manager (OWSM) secures Web services by using policy enforcement points. It has several types of policy enforcement points: Gateways, Web service client agents (client-Agents), and Web service agent (server-Agents). These policies are created, managed, and located in the OWSM Policy Manager, which is a centralized policy store.

An example policy could be to require authentication and authorization to all services. This policy is centrally stored in the Policy Manager, and is consumed by all the agents and gateways deployed in the SOA.

Gateway Use Case

An Internet facing Web service can be protected by inserting a gateway in front of it. A gateway is similar to a web based proxy, it is a choke point where all Web service traffic is forced through. Gateways are SOAP/XML intermediaries that enforce Web services policies while intermediating Web services traffic between clients and services.

This would be done using a WS-Security Username Token for authentication. Based on the credentials, the gateway should be able to authenticate and authorize access to the service.

Alternatively, WS-Security SAML Token can be applied when business partners expose business processes to each other within a secure context.

Agent Use Case

A Web service agent is another useful policy enforcement point. Agents (client-Agents and server- Agents) are SOAP interceptors that enforce Web services policies from within the same Web application.

Agents execute in the same process as the application, while Gateways run on separate processes and possibly on different servers. Gateways can manage services from multiple applications while agents control services belonging to the single application.

Web service is not exposed over the internet, then using agents is a viable option to secure the business process. There can be other reasons for using agents. For example, if one wants end-to-end security where the data is secured from the beginning to the end of the process.


OASIS WSS UsernameToken Profile (User name and password)

OASIS WSS X509 Token Profile 1.1 (certificate based)

OASIS WSS SAML Token Profile 1.1 (Xml accertation token from accertation server for sso)

OASIS WSS SWA Profile 1.1


Reference:

Web Services Security, Part 1

Web Services Security, Part 2

Web Services Security, Part 3

Apache WSS4J 1.1.0 Released

Securing Web Services and the Java WSDP 1.5


Tutorial

Securing Web Services Using the SAML or UserNameToken Profiles

Clustering and Securing Web Applications: A Tutorial


Web Services Security (2003)

http://www.xml.com/pub/a/ws/2003/03/04/security.html
http://www.xml.com/pub/a/ws/2003/04/01/security.html


Oracle TAF & FCF

Transparent Application Failover (TAF)

Transparent Application Failover (TAF) is a feature of the Java Database Connectivity (JDBC) Oracle Call Interface (OCI) driver. It enables the application to automatically reconnect to a database, if the database instance to which the connection is made fails. In this case, the active transactions roll back.

Transparent Application Failover (TAF) is a client-side feature that allows for clients to reconnect to surviving databases in the event of a failure of a database instance. Notifications are used by the server to trigger TAF callbacks on the client-side.

TAF is configured using either client-side specified TNS connect string or using server-side service attributes. However, if both methods are used to configure TAF, the server-side service attributes will supersede the client-side settings. The server-side service attributes are the preferred way to set up TAF.

TAF can operate in one of two modes, Session Failover and Select Failover. Session Failover will recreate lost connections and sessions. Select Failover will replay queries that were in progress.

Transparent Application Failover
Configuring Transparent Application Failover

Fast Connection Failover (FCF)

Fast Connection Failover offers a driver-independent way for your Java Database Connectivity (JDBC) application to take advantage of the connection failover facilities offered by Oracle Database 10g. The Fast Connection Failover mechanism depends on the implicit connection cache feature. As a result, for Fast Connection Failover to be available, implicit connection caching must be enabled.

FCF vs TAF

TAF is always active and does not have to be set, it does not work with the OCI Connection Pool.
Oracle recommends not to use TAF and Fast Connection Failover in the same application

Friday, October 26, 2007

Web Services Protocol Stack

You may hear web services for many years and already get some ideas of it, but always get confused and lost in the set of protocol and technology in use and not sure how they are related to each other in the picture.

Here, i would like to give another explanation of them from underlying protocol stack in use for web service so that you can organized, relate them properly in mind.

The Web service protocol stack is a collection of computer networking protocols that are used to define, locate, implement, and make Web services interact with each other. The Web service protocol stack mainly comprises four areas (source:wikipedia):

* Transport Protocol: responsible for transporting messages between network applications and includes protocols such as HTTP(s), SMTP, FTP, as well as the more recent Blocks Extensible Exchange Protocol (BEEP). Now, most common protocol in use is HTTP(S).

* Messaging Protocol: responsible for encoding messages in a common XML format so that they can be understood at either end of a network connection. Currently, this area includes such protocols as XML-RPC, WS-Addressing, REST and SOAP, which is most common one in use today.

* Description Protocol: used for describing the public interface to a specific web service. The WSDL interface format is typically used for this purpose.

* Discovery Protocol: centralizes services into a common registry such that network web services can publish their location and description, and makes it easy to discover what services are available on the network. At present, the UDDI API is normally used for service discovery.

Web service protocol stack also includes a whole range of recently defined protocols: BPEL, SOAP-DSIG.

In general terms, a web service is a software component or service that has been published on the World Wide Web in one of several particular ways, but industry tends to use the term web services more specifically, to describe services that are exposed using SOAP, an XML-based message encoding that is typically transported using HTTP.

XML-RPC

XML-RPC is a remote procedure call protocol which uses XML to encode its calls and HTTP as a transport mechanism. XML-RPC is a very simple protocol, defining only a handful of data types and commands, and the entire description can be printed on two pages of paper. Some people still prefer XML-RPC to SOAP because of its simplicity, minimalism, and ease of use. (source:wikipedia)

SOAP

SOAP is a protocol for exchanging XML-based messages over computer networks, normally using HTTP/HTTPS. SOAP forms the foundation layer of the Web services stack, providing a basic messaging framework so that more abstract layers can build on.

There are several different types of messaging patterns in SOAP, but by far the most common is the Remote Procedure Call (RPC) pattern, in which one network node (the client) sends a request message to another node (the server) and the server immediately sends a response message to the client.

SOAP is the successor of XML-RPC, though it borrows its transport and interaction neutrality and the envelope/header/body from elsewhere, probably from WDDX (source:wikipedia).

A SOAP message is an ordinary XML document containing the following elements:

* A required Envelope element that identifies the XML document as a SOAP message
* An optional Header element that contains header information
* A required Body element that contains call and response information
* An optional Fault element that provides information about errors that occurred while processing the message

All the elements above are declared in the default namespace for the SOAP envelope (source: w3schools)

WSDL

The Web Services Description Language (WSDL) is an XML-based language that provides a model for describing Web services.The current version of the specification is the 2.0. The WSDL defines services as collections of network endpoints, or ports. WSDL specification provides an XML format for documents for this purpose.A port is defined by associating a network address with a reusable binding, and a collection of ports define a service.

WSDL is often used in combination with SOAP and XML Schema to provide web services over the Internet. A client program connecting to a web service can read the WSDL to determine what functions are available on the server. Any special datatypes used are embedded in the WSDL file in the form of XML Schema. The client can then use SOAP to actually call one of the functions listed in the WSDL.WSDL 1.2 was renamed WSDL 2.0 because of its substantial differences from WSDL 1.1. By accepting binding to all the HTTP request methods (not only GET and POST as in version 1.1) WSDL 2.0 specification offers a better support for RESTful web services, much simpler to implement by beginners. (source:wikipedia)

A WSDL 1.1 document uses the following elements in the definition of network services:

* Types– a container for data type definitions using some type system (such as XSD).
* Message– an abstract, typed definition of the data being communicated.
* Operation– an abstract description of an action supported by the service.
* Port Type–an abstract set of operations supported by one or more endpoints.
* Binding– a concrete protocol and data format specification for a (not more than one) particular port type.
* Port– a single endpoint defined as a combination of a binding (not more than one) and a network address.
* Service– a collection of related endpoints (port).

WSDL 2.0 is different from WSDL 1.1 check out the example of a structured WSDL 2.0 document from here.

UDDI

Universal Description, Discovery and Integration (UDDI) is a platform-independent, XML-based registry for businesses worldwide to list themselves on the Internet. UDDI is a cross-industry effort driven by all major platform and software providers like Dell, Fujitsu, HP, Hitachi, IBM, Intel, Microsoft, Oracle, SAP, and Sun, as well as a large community of marketplace operators, and e-business leaders.

JAX-RPC
Check out my previous two posts (1|2) in this blog.

JAXR

Java API for XML Registries (JAXR) defines a standard API for Java platform applications to access and programmatically interact with various kinds of metadata registries. JAXR provides a uniform and standard Java API for accessing different kinds of XML-based metadata registry.

Current implementations of JAXR support ebXML Registry version 2.0, and UDDI version 2.0. More such registries could be defined in the future.

The JAXR API insulates application code from the underlying registry mechanism. When writing a JAXR based client to browse or populate a registry, the code does not have to change if the registry changes, for instance from UDDI to ebXML.

WS-I
The WS-I Basic Profile (official abbreviation is BP), a specification from the Web Services Interoperability industry consortium (WS-I), provides interoperability guidance for core Web Services specifications such as SOAP, WSDL, and UDDI. The profile uses Web Services Description Language (WSDL) to enable the description of services as sets of endpoints operating on messages. BP complied web service means smooth internal operability (messaging, description and registry) between web services regardless of underlying implementation.

Thursday, October 25, 2007

Open Source Tool for JDBC Debugging and Monitoring

P6Spy is a open source framework to intercept and debug JDBC interaction with a database and optionally modify database statements. It's a wrapper of all JDBC elements ( Connection, PreparedStatement, ResultSet... ) and it has a powerful feature in order to log all informations about those interactions. P6Spy includes P6Log, an application that intercepts and logs the database statements of any application that uses JDBC.

This application is particularly useful for developers to monitor the SQL statements produced by EJB servers, enabling the developer to write code that achieves maximum efficiency on the server.

You can also use it together with spring and hibernate for performance tuning as well.

Reference List:

P6Spy

P6Spy Official Home

Integrate P6Spy with Spring

Improving Hibernate's Performance

P6Spy released: automatic logging of your database/EJB queries

XML-RPC vs JAX-RPC

XML-RPC

XML-RPC is a remote procedure call protocol which uses XML to encode its calls and HTTP as a transport mechanism. XML-RPC is a very simple protocol, defining only a handful of data types and commands, and the entire description can be printed on two pages of paper. Some people still prefer XML-RPC to SOAP because of its simplicity, minimalism, and ease of use. (source wikipedia)

The original XML-RPC uses only a few data types and does not require namespaces. It achieves considerable flexibility by defining and types which can contain other values. Responses are also simple XML with as the root element with either or as the child element. The contrast with complex SOAP responses is striking.

For xml-rpc, wsdl and uddi is not involved for client to invoke the web service.

Implementations of XML-RPC:

There are five Implementations in Java list at xmlrpc.com as of December 2003. However, many of these projects appear to be inactive. The project hosted by the Apache Web Services Project is active. It looks like the Apache Web Services Project is on the right track with version 3, so keep an eye open on it if you are interested in an open-source XML-RPC solution.

JAX-RPC - The Java API for XML-based RPC

Instead of using xml-rpc protocol, JAX-RPC make used of SOAP protocol for messaging which enable web service a industry standard.

SOAP

SOAP is a protocol for exchanging XML-based messages over computer networks,normally using HTTP/HTTPS. SOAP forms the foundation layer of the Web services stack, providing a basic messaging framework so that more abstract layers can build on.

There are several different types of messaging patterns in SOAP, but by far the most common is the Remote Procedure Call (RPC) pattern, in which one network node (the client) sends a request message to another node (the server) and the server immediately sends a response message to the client. SOAP is the successor of XML-RPC, though it borrows its transport and interaction neutrality and the envelope/header/body from elsewhere, probably from WDDX. (Source: wikipedia)

WSDL is involved for client to invoke the web service when use JAX-RPC.

Is XML-RPC Still Useful?

The complexity of the JAX-RPC implementation has caused somewhat of a backlash among programmers. It seems you should consider XML-RPC over JAX-RPC if:

* Your application uses relatively simple requests
* You control both server and client sides and/or you don't need to publish a WSDL
* You want to keep client-side parsing simple (as in AJAX applications)

Wednesday, October 24, 2007

Architecture Chaos of Web 2.0 Application

Struts, Spring, Seam, JSF, Hibernate, iBatis, Ajax and EJB, there are so many java technology, my question is which one should go for to mix and match out a wonderful J2EE Web 2.0 architecture for new project.


Model layer


Spring+Hibernate
Spring+iBatis (With skillful DBA or developers with advnaced sql skill)
EJB 3.0, forget about the EJB 2.1 (with developers expert in JDK 1.5 and annotation)

View & Controller layer


Struts1 + Ajax (zk/echo/gwt) + Tile (templating)
Spring MVC + Ajax (zk /echo/gwt) + Tile (templating)
JSF+Icefaces/Zk(AJAX)+Facelet / Tile(templating)
RichFaces / AJAX4JSF

Integration Layer

Web services
BPEL process (a collection of inter-related web services)
Enterprise Service Bus
JMS & MOM

Other than suggestion above, JBoss seam and Oracle ADF and Webcenter come with full technology stack for web 2.0 architecture.

As for which technology and plaftform should be used in the development, not only it depends on the value and benefits technology bring to the project, but available tools and people to support and accelerate the development play a part as well.

Only with good tools (development & testing) and big pool of talent people/developer, project can be delivered on time.

So, which tools should be chosen for the project development? Again, there are many open tools available in each phase of SDLC.

Someone can use a combination of tools below:

Eclipse with Checkstyle, PMD, Mylar, Subversion (Subclipse), JUnit plug-ins
Subversion version control
maven2
Jira issue tracking
Confluence wiki
Cobertura test coverage
QuickBuild Continuous Integration system
Enterprise Architect for requirements and design in UML, and document generation

But it may not fit your project and you will need to figure out your own tool set with some R&D work on it.

Here, i would like to suggest a book (not-yet-published, but coming-soon) called "Java power tools". It maintain a comprehensive open tools and I believe with this book, people can shorten the R&D work for development tools findings.

So, what is your voice for this confusion and chaos?

Monday, October 22, 2007

JAXM & Messaging

JAXM Definition

JAXM, the Java API for XML Messaging, defines a lightweight Java API for composing, processing, sending, and receiving XML documents. It supports both synchronous (HTTP) and asynchronous (SMTP) messaging and “Out-of-the-box” support for ebXML messaging.

Fundamental Elements

The JAXM Service

A JAXM service consumes JAXM messages sent by a JAXM client. To develop a JAXM service, developers extend the class javax.xml.messaging.JAXMServlet and implement either the javax.xml.messaging.OnewayListener or javax.xml.messaging.ReqRespListener interface. The choice of interface depends upon whether the interaction between the client and the server is asynchronous (OnewayListener) or synchronous (ReqRespListener) in nature.

The JAXM Client

Clients exchange messages with JAXM services. JAXM clients can either interact directly with a JAXM service or go through a JAXM provider. JAXM clients that interact directly with a JAXM service can only participate in synchronous interactions. JAXM clients that use a messaging provider can participate in asynchronous as well as synchronous interactions with the JAXM service.

The JAXM Messaging Provider

A JAXM messaging provider is responsible for managing the routing of messages for a JAXM client. Besides providing a degree of separation between a JAXM client and service, a provider can also offer additional services like reliable messaging, security, and transaction support. Currently, clients that want to interact asynchronously with a JAXM service have to use a provider.

The JAXM Message

All JAXM messages conform to the SOAP 1.1 and SOAP with Attachments standards. JAXM messages also support the concept of higher-level protocols like ebXML through the concept of messaging profiles. A messaging profile defines the messaging contract between two parties. A JAXM client and service that share a message profile like ebXML can conduct business by exchanging ebXML messages.

JAXM & Messaging

* JAXM is an synchronous / asynchronous (queue-based) messaging API
* Not part of J2EE 1.4
* Uses queues on either side
* Doesn’t provide WSDL facilities
* Can be used in conjunction with JAXB to convert to objects

Difference in JAXM & JMS

The JAXM API was designed for Internet SOAP-based messaging, while the JMS API was designed as a common messaging API to be layered over existing enterprise message-oriented middleware (MOM) applications.

The delivery endpoint models differ. JAXM provides direct point-to-point messaging, while JMS uses destinations to provide a reliable asynchronous messaging capability.

JMS clients send messages to, and retrieve them from, destinations. JMS provides a model whereby an intermediary, a messaging server, makes sure a message is delivered between two applications that do not know directly about each other, but know only about the common destination.

A JAXM client is interoperable over any SOAP 1.1 compatible client (there must be an agreed-upon profile between providers, however), whereas a JMS client is interoperable only with other JMS clients over the same messaging system (or another messaging system for which the JMS provider supports a proprietary gateway).

The JMS API does not define interoperability between message providers. It does not define a common message format, only a number of message types; the clients must agree on the precise format. (It is possible to wrap a SOAP message, or any other kind of XML message, in a TextMessage and send it via JMS.)

Find out more info from JAXM FAQ

Why there is a need for JAXM?

The goal of JAXM is to provide a rich set of interfaces for document-style web services. Document-style web services enable the exchange of XML documents between two parties, as opposed to RPC-style web services, which expose software functions as web services.

JAXM-based web services can be used effectively in Such scenario:

When two parties want to exchange data using XML documents that are bounded using well-defined XML schemas instead of invoking software functions (Java objects, C procedures, etc.) exposed as RPC-style web services.

I would like to recommend you to go through a article called "Developing E-Business Interactions with JAXM" for a good understanding of How & Where to use JAXM

Implementation

Sun provides JAXM implementation, but does not support full ebXML messaging service specification, there is alternative implementation of ebMS 2.0 from www.freebxml.org.

ebXML & Message Service

Find more from my previous post here.
Find more from article called "ebXML Message Service - An In-depth Overview"

Reference List:

http://java.sun.com/webservices/jaxm
Sun JAXM tutorial
Developing E-Business Interactions with JAXM

JAX-RPC vs JAX-WS

Java API for XML-Based RPC (JAX-RPC) is a Legacy Web Services Java API, it uses SOAP and HTTP to do RPCs over the network and enables building of Web services and Web applications based on the SOAP 1.1 specification, Java SE 1.4 or lower, or when rpc/encoded style must be used.

You can use the JAX-RPC programming model to develop SOAP-based web service clients and endpoints. JAX-RPC enables clients to invoke web services developed across heterogeneous platforms. Likewise, JAX-RPC web service endpoints can be invoked by heterogeneous clients. JAX-RPC requires SOAP and WSDL standards for this cross-platform interoperability.

JAX-RPC lets people develop a web service endpoint using either a Servlet or Enterprise JavaBeans (EJB) component model. The endpoint is then deployed on either the Web or EJB container, based on the corresponding component model. Endpoints are described using a Web Services Description Language (WSDL) document.(This WSDL document can be published in a public or private registry, though this is not required). A client then uses this WSDL document and invokes the web service endpoint.

JAX-RPC in J2ee 1.4 supports 4 types of stubs and invocations: static stub, dynamic proxy, Dynamic Invocation Interface (DII) and Application client.

Static Stub Client
Web service client makes a call through a stub, a local object that acts as a proxy for the remote service. Because the stub is created by wscompile at development time (as opposed to runtime), it is usually called a static stub.

Example: Invoking a Stub Client
Stub stub = (Stub) (new MyHelloService_Impl().getHelloIFPort());
stub._setProperty (javax.xml.rpc.Stub.ENDPOINT_ADDRESS_PROPERTY,endpoint_address_string);
HelloIF hello = (HelloIF)stub;
System.out.println(hello.sayHello("Duke!"));

Dynamic Proxy Client
In contrast, the client call a remote procedure through a dynamic proxy, a class that is created during runtime. Although the source code for the static stub client relies on an implementation-specific class, the code for the dynamic proxy client does not have this limitation.

Example: Dynamic Proxy
javax.xml.rpc.Service service = ServiceFactory.newInstance().createService(...);
com.example.StockQuoteProvider sqp = (com.example.StockQuoteProvider)service.getPort(portName, StockQuoteProvider.class);
float price = sqp.getLastTradePrice("ACME");

Dynamic Invocation Interface Client
With the dynamic invocation interface (DII), a client can call a remote procedure even if the signature of the remote procedure or the name of the service is unknown until runtime. In contrast to a static stub or dynamic proxy client, a DII client does not require runtime classes generated by wscompile.

Example: Dynamic Invocation Interface
javax.xml.rpc.Service service = ServiceFactory.newInstance().createService(...);
javax.xml.rpc.Call call = service.createCall(portName, "getLastTradePrice");
// This example assumes that addParameter and setReturnType methods are not required to be called
Object[] inParams = new Object[] {"ACME"};
Float quotePrice = (Float)call.invoke(inParams);

Application Client
Unlike the stand-alone clients, for an application client, because it's a J2EE component, an application client can locate a local web service by invoking the JNDI lookup method.

Example: Application Client
Context ic = new InitialContext();
MyHelloService myHelloService = (MyHelloService)
ic.lookup("java:comp/env/service/MyJAXRPCHello");
appclient.HelloIF helloPort = myHelloService.getHelloIFPort();
((Stub)helloPort)._setProperty(Stub.ENDPOINT_ADDRESS_PROPERTY,args[0]);
System.out.println(helloPort.sayHello("Jake!"));

Service Endpoint Model

JAX-RPC supports a client model for the service consumer, and a service endpoint model for the service producer.

Application-Level Interaction Modes

JAX-RPC specifies three client application interaction models:
* Synchronous request-response two-way RPC
* Asynchronous (non-blocking) request-response two-way RPC
* One-way RPC

JAX-WS 2.0 is the successor of JAX-RPC 1.1 - the Java API for XML-based Web services. If possible, JAX-WS should be used instead as it is based on the most recent industry standards.

What remains the same?

Before we itemize the differences between JAX-RPC 1.1 and JAX-WS 2.0, we should first discuss what is the same.

* JAX-WS still supports SOAP 1.1 over HTTP 1.1, so interoperability will not be affected. The same messages can still flow across the wire.

* JAX-WS still supports WSDL 1.1, so what you've learned about that specification is still useful. A WSDL 2.0 specification is nearing completion, but it was still in the works at the time that JAX-WS 2.0 was finalized.

What is different?

* SOAP 1.2
JAX-RPC and JAX-WS support SOAP 1.1. JAX-WS also supports SOAP 1.2.

* XML/HTTP
The WSDL 1.1 specification defined an HTTP binding, which is a means by which you can send XML messages over HTTP without SOAP. JAX-RPC ignored the HTTP binding. JAX-WS adds support for it.

* WS-I's Basic Profiles
JAX-RPC supports WS-I's Basic Profile (BP) version 1.0. JAX-WS supports BP 1.1. (WS-I is the Web services interoperability organization.)

* New Java features
o JAX-RPC maps to Java 1.4. JAX-WS maps to Java 5.0. JAX-WS relies on many of the features new in Java 5.0.
o Java EE 5, the successor to J2EE 1.4, adds support for JAX-WS, but it also retains support for JAX-RPC, which could be confusing to today's Web services novices.

* The data mapping model
o JAX-RPC has its own data mapping model, which covers about 90 percent of all schema types. Those that it does not cover are mapped to javax.xml.soap.SOAPElement.
o JAX-WS's data mapping model is JAXB. JAXB promises mappings for all XML schemas.

* The interface mapping model
JAX-WS's basic interface mapping model is not extensively different from JAX-RPC's; however:
o JAX-WS's model makes use of new Java 5.0 features.
o JAX-WS's model introduces asynchronous functionality.

* The dynamic programming model
o JAX-WS's dynamic client model is quite different from JAX-RPC's. Many of the changes acknowledge industry needs:
+ It introduces message-oriented functionality.
+ It introduces dynamic asynchronous functionality.
o JAX-WS also adds a dynamic server model, which JAX-RPC does not have.

* MTOM (Message Transmission Optimization Mechanism)
JAX-WS, via JAXB, adds support for MTOM, the new attachment specification. Microsoft never bought into the SOAP with Attachments specification; but it appears that everyone supports MTOM, so attachment interoperability should become a reality.

* The handler model
o The handler model has changed quite a bit from JAX-RPC to JAX-WS.
o JAX-RPC handlers rely on SAAJ 1.2. JAX-WS handlers rely on the new SAAJ 1.3 specification.

Additional reading list:

Developing Web Services at a Glance
http://java.sun.com/webservices/technologies/index.jsp#Core_Web_Services
Java API for XML-based RPC (JAX-RPC)
http://java.sun.com/developer/technicalArticles/xml/jaxrpc/

Reference:
http://java.sun.com/webservices/technologies/index.jsp#Core_Web_Services
http://www-128.ibm.com/developerworks/webservices/library/ws-tip-jaxwsrpc.html
http://www.ibm.com/developerworks/webservices/library/ws-tip-jaxwsrpc2.html

Keywords: SAAJ, JAXB, SOAP, JAX-RPC, JAX-WS

Using Oracle RAC in OAS, Weblogic and Websphere

Oracle Application Server 10g R2 (OAS)

To enable the Fast Connection Failover mechanism, properties of "connectionCachingEnabled" and "fastConnectionFailoverEnabled" must be set to true on the (connection-factory) tag for an OracleDataSource object. the JDBC URL String must be set using the service name syntax. The service name specified on the connection URL is used to map the connection cache to the service.

Valid URL Usage

url="jdbc:oracle:oci:@TNS_ALIAS"

url="jdbc:oracle:oci:@(DESCRIPTION=
(LOAD_BALANCE=on)
(ADDRESS=(PROTOCOL=TCP)(HOST=host1) (PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=host2)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=service_name)))"

url="jdbc:oracle:oci:@(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=cluster_alias) (PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=service_name)))"

url = "jdbc:oracle:thin@//host:port/service_name"

url = "jdbc:oracle:thin@//cluster-alias:port/service_name"

url="jdbc:oracle:thin:@(DESCRIPTION=
(LOAD_BALANCE=on)
(ADDRESS=(PROTOCOL=TCP)(HOST=host1) (PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=host2)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=service_name)))"

url = "jdbc:oracle:thin:@(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=cluster_alias) (PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=service_name)))"


If a SID is specified on the URL, when Fast Connection Failover is enabled, then an exception is thrown.

For more inforamtion, find them from links below:

http://download.oracle.com/docs/cd/B31017_01/web.1013/b28958/datasrc.htm#CHDEHFHE
http://download.oracle.com/docs/cd/B31017_01/core.1013/b28939/j2ee.htm#ASTED006
http://download.oracle.com/docs/cd/B31017_01/core.1013/b28939/security.htm#CDDGFADE


Weblogic

For how to use Oracle RAC DB with Weblogic application server, find the detail information from links below:

http://edocs.bea.com/wls/docs81/jdbc/oracle_rac.html
http://edocs.bea.com/platform/docs81/db_mgmt/oracle_rac.html

Websphere

For how to use Oracle RAC DB with Websphere application server, find the detail information from links below:

http://www.ibm.com/developerworks/websphere/techjournal/0710_barghouthi/0710_barghouthi.html
http://www.ibm.com/developerworks/websphere/techjournal/0706_banerjee/0706_banerjee.html

JAXP Implementation Provider

The Java API for XML Processing (JAXP) enables applications to parse (SAX/DOM), transform (XSLT), validate (Against the XML schema) and query (XPath) XML documents using an API that is independent of a particular XML processor implementation. JAXP provides a pluggability layer to enable vendors to provide their own implementations without introducing dependencies in application code. JAXP selects the parser based on system properties. You can set the properties to select a different parser than the default one.

Existing JAXP implementation include, but not limited to Resin, Xalan, Xerces and Crimson.

Xerces and Crimson is an XML parser. Xalan is an implementation of XPath and XSLT. Xalan can work with any JAXP-compliant XML parser. Resin has parser and transformer implementation for all JAXP API.

You can find the more information on each JAXP implementation from links below:

XML Parsing
http://xml.apache.org/crimson/ (sun default jaxp implementation)
http://xerces.apache.org/xerces-j/
http://www.caucho.com/resin-3.0/xml/jaxp.xtp

XMl Transformation and XPath

http://xalan.apache.org/
http://www.caucho.com/resin-3.0/xml/jaxp.xtp

Additional jaxp reading list (Past, Current and Future Roadmap)

The Evolution of JAXP
http://www.xml.com/pub/a/2005/07/06/jaxp.html?page=1

Friday, October 19, 2007

Jakarta Common Validator & Struts Validator

The Commons Validator is a basic validation framework that lets you define validation rules for a JavaBean in an xml file. Validators, the validation definition, can also be defined in the xml file. An example of a validator would be defining what method and class will be called to perform the validation for a required field. Validation rules can be grouped together based on locale and a JavaBean/Form that the rules are associated with. The framework has basic support for user defined constants which can be used in some field attributes.

Here
is the quick tutorial for Jakarta common validator, you can see that there is a need to define a validation rule xml file to declare the validation rules and objects & fields to be validated.

Noted, inside the xml file, the individual fields in the input data are called fields, the set of input data is collectively called a form, and a collection of forms is called a form-set.

The relationship between the elements is shown here:
Fields --> Forms --> FormSet

Struts Validator is built on Jakarta Commons validator and other common packages. It split the rule defintion xml file into two, one (validation-rules.xml) is for rule definitions, another (validation.xml) is for which and what object and fields to be validated. Struts validator is a web form validator and support both client and server side validation.

Here is a tutorial for how to use the Struts Validator.

Tuesday, October 16, 2007

EJB Reference Definiton and Lookup in Oracle Application Server 10g

EJB Reference Definition

There are two way for EJB reference definition in Oracle AS:

One is define the (ejb-ref-name) together with (ejb-link) within the (ejb-ref) element. Configure (ejb-ref-name) with a logical name and configure (ejb-link) with the actual name of the target bean as shown below:

Example: Configuring ejb-ref-name with a Logical Name Resolved by ejb-link

(ejb-ref)
(ejb-ref-name)ejb/nextVal(/ejb-ref-name)
(ejb-ref-type)Session(/ejb-ref-type)
(home)myBeans.BeanAHome(/home)
(remote)myBeans.BeanA(/remote)
(ejb-link)myBeans/BeanA(/ejb-link)
(/ejb-ref)

This option provides indirection that offers assembly and deployment flexibility.

Another way is to Configure (ejb-ref-name) with a logical name and in the orion-ejb-jar.xml deployment descriptor, define an (ejb-ref-mapping) element that maps the logical name to the actual name of the target bean as shown below:

Example: Configuring ejb-ref-name with a Logical Name Resolved by ejb-ref-mapping

(ejb-ref)
(ejb-ref-name)ejb/nextVal(/ejb-ref-name)
(ejb-ref-type)Session(/ejb-ref-type)
(home)myBeans.BeanAHome(/home)
(remote)myBeans.BeanA(/remote)
(/ejb-ref)

Example: Mapping Logical Name to Actual Name with ejb-ref-mapping

(ejb-ref-mapping name="ejb/nextVal" location="myBeans/BeanA"/)

This option provides indirection that offers the most assembly and deployment flexibility.


EJB Reference Lookup

To lookup EJB in the OAS, there are two ways too. One is using the ejb-ref (logical name in the web.xml/ejb-jar.xml) and second way is using the real location (jndi name in the server).

For sample code snippet, refer to links below:

http://download.oracle.com/docs/cd/B25221_04/web.1013/b14428/servjndi.htm#CHDGBAFJ
http://download.oracle.com/docs/cd/B25221_04/web.1013/b14427/jndi.htm#i1084614
http://download.oracle.com/docs/cd/B25221_04/web.1013/b14427/jndi.htm#CIHGAHFF

Oracle Application Server 10g How-To's

This link provides a high-level overview of a feature along with relevant code excerpts to illustrate how the feature can be used.

Infrastructure
How-To Use Application Properties
How-To Register MBeans on Container Startup
How-To Create Simple JMX MBeans Mar-2006
How-To Swap Oracle JDBC Drivers
How-To Swap XML Parsers
How-To Configure and Use Job Scheduler with database persistence
How-To Configure and Use Job Scheduler with JMS persistence
How-To Configure and Use JCA 1.5 Resource Adapters
How-To Configure and Use Data Sources
How-To Configure and Use JNDI
How-To Configure and Use Proxy-authentication with Data Sources
How to Configure and use Active MQ JCA 1.5 Resource Adapter with OC4J 10 g (10.1.3)
How-To Use ORMI/SSL with OC4J
How-To Propagate a transaction context between OC4J instances
How-to Stop an MDB via JMX (Blog)
How-to Establish a Remote connections to OC4J with JConsole (Blog)

Servlets and JSP / Web Container

How-To Use Dependency Injection in a Web Module
How-To Use Servlet 2.4 Features
How-To Use HTTPClient in OC4J 10g

Web Services
How-To Use MTOM to exchange binary data
How-To Develop a Web Service From an Existing Java Class
How-To Develop a Web Service From a WSDL Document
How-To Generate a Web service using Web service Metadata Annotations (JSR 181)
How -To Use WS Security between OracleAS Web Services and Microsoft .Net
How to Use JDeveloper 10.1.3 to Secure and Test a Web Service
How-to Share State of Stateful Web Services Between Clients (Blog)
How to use SOAP Compression using JAX-RPC, on OC4J (Blog)
How to use SSL with OracleAS Web Services (Blog)

Spring Framework
Using the New Spring JTA Transaction Manager
Use EJB 3.0 and Java Persistence API with Spring in OC4J
Spring-TopLink Pet Clinic Demo

Messaging (JMS)
How-To Use the OracleAS JMS Router
How to Configure and Use Oracle's Generic JMS Resource Adapter
With IBM WebSphere MQ JMS
With Tibco Enterprise for JMS
With SonicMQ JMS
With OEMS JMS (In-Memory and File-Based)
With OEMS JMS (Database)
How-to Create as Simple MDB with Oracle Database JMS Provider (Blog)
How-to Propagate JMS Messages to Multiple Targets (Blog)
How-to use OracleAS JMS, Hermes and MBeans (Blog)

J2EE Security / JAAS
How-To Integrate a Custom JAAS Login Module in OracleAS
How-To Reset oc4jadmin password (Blog)

Oracle Rules Engine
How-To Build a Rules Enabled Java Application
How-To build a Rules Enabled Application using XML facts

Additional Resources
Tutorial for Java EE Developers (10.1.3.1.0)
How to use SASH Starter 2.0 with OracleAS 10g R3
EJB 3.0 How-To's
Oracle TopLink How-To's
Oracle JDeveloper How-To's
How-To Use Eclipse with OC4J 10g (Eclipse Web Tools Project Page)

Friday, October 5, 2007

Where did System.out | System.err statements print in Oracle AS 10g (10.1.3)?

By default, Sys.out statements are printed in the $Oracle_Home/opmn/logs/_default_island_# folder. But we can change the location by modify the opmn.xml file.

To specify other out file location:

We use "-out [file]" option in the oc4j startup parameter list to Specifies a file to route the standard output to. The specified file contains messages that are printed to System.out, as well as the messages sent to output through the servlet logging interface. If not specified, all output is written to standard out.

Note that in an OPMN-managed configuration, the file will be generated within an instance_default_group_1 directory appended to the path specified.

For example, if you specify the following in opmn.xml:


The mylog.log file will actually be generated in:

/mypath/instanceName_default_group_1/mylog.log.

To specify other err file location:

We use "-err [file]" option in the oc4j startup parameter list to specify a file to route standard error output to. The file contains messages that are printed to System.err. If not specified, all errors are written to standard error.

Note that in an OPMN-managed configuration, the file will be generated within an instance_default_group_1 directory appended to the path specified. See the -out description above for details.

See "Managing stdout/stderr Log Files" for additional system properties that can be set to manage stdout files.

Thursday, October 4, 2007

TIBCO Rendezvous vs SonicMQ

This article is compiled from source available in the WWW, not my own opinion and only used for reference and information purpose.

TIBCO Rendezvous

TIBCO has been one of the leading providers of EAI since its establishment 20 years ago and TIB/RV is one of the most widely used messaging middleware in enterprises.

TIB/RV is implemented in C and based on a distributed architecture. An installation of TIB/RV resides on each host on the network.

Messaging can be delivered in point-to-point or publish/subscribe, synchronously or asynchronously, locally delivered or sent via WAN or the Internet. Rendezvous messages are self-describing and platform independent.

TIB/RV is composed of three main components:
- RV Daemon (RVD) responsible for the delivery of messages within a LAN.
- RV Agent (RVA)
- RV Routing Daemon (RVRD)

In TIB/RV, point-to-point messages sending model is fairly similar to publish/subscribe, The RV Sender program passes the message and destination topic to RVD. RVD then broadcasts this message using User Data Packet (UDP) to the entire network. All subscribing computers with RVDs on the network will receive this message. RVD will filter the messages which non-subscribers will not be notified of the message. Therefore only subscriber programs to the particular topic will get the messages.

SonicMQ

SonicMQ is JMS implementation, and has outstanding performances competitive with existing MOM technologies, such as IBM MQSeries. SonicMQ is written in 100% pure Java, supports XML messaging, and HTTP tunnelling to allow SonicMQ to work over the Internet. The underlying mechanism of SonicMQ is its “broker” that facilitates the movement of messages across the network.

The communication protocols that can be used with SonicMQ include TCP, HTTP and SSL. Since it uses common Internet protocol, SonicMQ can extend its deployment to the Internet. It also provides bridges to many other popular MOMs that allow messages to be sent and received between SonicMQ and other MOMs.

There are three types of configurations a user can choose from:
- Single-broker Configuration: Under this configuration, there is one broker which is being shared across a few nodes.
- Multi-broker Clusters
- Multi-node Configurations

The broker is the most important underlying implementation of SonicMQ. It is responsible for delivering and acquiring of messages within a LAN environment. It is a client-server model, where many clients connect to a single broker. The connection can be via TCP (for LAN), SSL (for security encryption), or even HTTP (to connect to external entities).

The downside with single broker configuration is that scalability is limited by the capabilities of the node machine. Also the system is dependent on the single broker machine (node), hence leading to a bottleneck of the system at the node. The whole system may collapse if the node goes down. To solve this problem a multi-broker cluster must be used.

There exist a benchmark report for SonicMQ by Progress Software. SonicMQ showed outstanding performances compared to IBM MQSeries and Fiorano FioranoMQ, (both are JMS implementations) under WinNT platform.

TIB/RV vs SonicMQ

There is a benchmarking result shows that TIB/RV has exceptional performance compared to SonicMQ.

They are as follows:
- High publish/subscribe and point-to-point send/receive rates
- High scalability:
- Publishing rate not affected by introducing more receivers;
- Subscriber rate increases as more subscribers are introduced.
- Low memory and CPU consumption

The only major downside of TIB/RV is that when there are very few receivers in the network, it could flood the network with many unnecessary UDP packets, introducing congestions.

TIB/RV is using multicast-based publish/subscribe messaging, which is an excellent solution for near-real-time message dissemination when 1 to 'very-many' delivery capabilities matter. It is better for application with high volume, high performance, acceptance of unreliability (nature of UDP), 1 to very many.

However, in most EAI cases the opposite requirements are true: ‘Cardinality’ of message delivery is 1-1, 1-2; 1 to-very-many is a rare case. With exception of ‘consolidated application’ integration model (near real time request reply with timeout heuristics), reliability of message delivery takes priority over performance.

Reference List

Benchmarking Message-Oriented Middleware TIB/RV vs SonicMQ
Tibco RV vs Tibco EMS

Wednesday, October 3, 2007

Hands-on Tutorial of Oracle BI EE

1. BI EE Installation http://www.oracle.com/technology/obe/obe_bi/bi_ee_1013/install/installoraclebee.htm

2. Create A Repository
http://www.oracle.com/technology/obe/obe_bi/bi_ee_1013/bi_admin/biadmin.html

3. Create A DashBoard http://www.oracle.com/technology/obe/obe_bi/bi_ee_1013/saw/saw.html

4. Integration with BI Publisher http://www.oracle.com/technology/obe/obe_bi/bipub/bip_biee/bip_biee.htm

OU Reference
http://ouweb.us.oracle.com/database/oracle_business_inteligence_series_replays.html

BI Primer

Recently, i am looking at the Oracle Business Intelligence (BI) solution and preparing a technical sharing session.

When I dive into the BI, i found it is a damn big topic and a lot of acronym and technical term need to be clear before get to understand it.

Information here is compiled from multiple source available from the internet to clear those prerequisite.

What is BI ?

Business intelligence (BI) is a business management term, which refers to applications and technologies that are used to gather, provide access to, and analyze data and information about company operations.

BI systems can help companies have a more comprehensive knowledge of the factors affecting their business, such as metrics on sales, production, internal operations, and they can help companies to make better business decisions.

BI software and applications includes a range of tools. Some BI applications are used to analyze performance, projects, or internal operations. Other BI applications are used to store and analyze data, such as Data mining (DM), Data Farming, and Data warehouses; Decision Support Systems (DSS) and Forecasting. Other BI applications are used to analyze or manage the "human" side of businesses, such as Customer Relationship Management (CRM) and Marketing tools and Human Resources applications.

BI often uses key performance indicators (KPIs) to assess the present state of business and to prescribe a course of action. The KPI methodology was further expanded with the Chief Performance Officer methodology which incorporated KPIs and root cause analysis into a single methodology.


What is OLTP ?

OLTP stand for Online Transaction Processing. This is a standard, normalized database structure. OLTP is designed for transactions, which means that inserts, updates, and deletes must be fast. Imagine a call center that takes orders. Call takers are continually taking calls and entering orders that may contain numerous items. Each order and each item must be inserted into a database. Since the performance of the database is critical, we want to maximize the speed of inserts (and updates and deletes). To maximize performance, we typically try to hold as few records in the database as possible. OLTP is typically for data entry and retrieval transaction processing.


What us OLAP ?

Online Analytical Processing is an approach to quickly providing answers to analytical queries that are multidimensional in nature. OLAP is part of the broader category BI, which also includes Extract transform load (ETL), relational reporting and data mining. The typical applications of OLAP are in business reporting for sales, marketing, management reporting, business process management (BPM), budgeting and forecasting, financial reporting and similar areas. The term OLAP was created as a slight modification of the traditional database term OLTP (Online Transaction Processing).

The output of an OLAP query is typically displayed in a matrix (or pivot) format. The dimensions form the row and column of the matrix; the measures, the values.

What is an OLAP cube ?

The term "cube" comes from the geometric object that has three dimensions. OLAP cubes can have many more dimensions than 3, but the term continues to apply.

In database theory, an OLAP cube is an abstract representation of a projection of an RDBMS relation. It is an arrangement of data in arrays (alternatively called cubes) to allow fast analysis. The arrangement of data into cubes avoids a limitation of relational databases which are not well suited for near instantaneous analysis of large amounts of data. OLAP cubes can be thought of as extensions to the two-dimensional array of a spreadsheet. Because there can be more than three dimensions in an OLAP system the term hypercube is sometimes used. The commercial OLAP products have different methods of creating the cubes and hypercubes and of linking cubes and hypercubes.

What is Measure, Dimension & Hierarchy?

While most databases designed for OLTP such as those used in claims processing are designed for efficiency in data storage, OLAP cubes are designed for efficiency in data retrieval.

Regular relational databases treat all data into the database similarly, however OLAP cubes categorize data into "dimensions" and "measures".

Measures represent items that are counted, summarized or aggregated, such as costs or units of service.

A company might wish to analyse some financial data by product, by time-period, by city, by type of revenue and cost, and by comparing actual data with a budget. These additional methods of analysing the data are known as dimensions.

Dimensions are variables by which measures are summarized, such as hospitals, physicians, or dates of service. This organization of data greatly facilitates the ability to formulate data requests based on real-life situations.

Each of the elements of a dimension could be summarised using a hierarchy. For example May 2005 could be summarised into Second Quarter 2005 which in turn would be summarised in the Year 2005. Similarly the cities could be summarised into regions, countries and then global regions; products could be summarised into larger categories; and cost headings could be grouped into types of expenditure. Conversely the analyst could start at a highly summarised level such as the total difference between the actual results and the budget and drill down into the cube to discover which locations, products and periods had produced this difference.

In addition, many of queries that could be posed to the data are "pre-aggregated" in the database such that the answers have already been precalculated and can be reported without delay.

What is Oracle OLAP Cube?

Oracle OLAP Cube requires a specific set of materialized views to map to a star schema. A materialized view is a replica of a target master from a single point in time. The master can be either a master table at a master site or a master materialized view at a materialized view site. ( more about 1 | 2).

Why are OLAP cubes important ?

Before OLAP technology was well developed, data had to be extracted from databases using "queries".

This meant that the analyst had to structure a request to the database for the information desired, and then submitted this query to the database server. That server would processing query and return the results.

Depending on the size of the database and the data requested, this query could take minutes or hours to complete.

In this sense, the "online" aspect of this type of reporting is questionable. OLAP cubes are fundamentally different in that they "pre-aggregate" the data used to answer many of queries that are anticipated.


What is star schema?

The star schema (sometimes referenced as star join schema) is the simplest style of data warehouse schema, consisting of a few "fact tables" (possibly only one, justifying the name) referencing any number of "dimension tables". The "facts" that the data warehouse helps analyze are classified along different "dimensions": the fact tables hold the main data, while the usually smaller dimension tables describe each value of a dimension and can be joined to fact tables as needed.

Dimension tables have a simple primary key, while fact tables have a compound primary key consisting of the aggregate of relevant dimension keys.

Facts and Dimensions

When we talk about the way we want to look at data, we usually want to see some sort of aggregated data. These data are called measures. These measures are numeric values that are measurable and additive. For example, our sales dollars are a perfect measure.
Just tracking measures isn't enough, however. We need to look at our measures using those "by conditions". These "by conditions" are called dimensions.

Therefore, in designing a star schema, our first order of business is usually to determine what we want to see (our measures) and how we want to see it (our dimensions).

Building the Fact Table

The Fact Table holds our measures, or facts. The measures are numeric and additive across some or all of the dimensions. For example, sales are numeric and we can look at total sales for a product, or category, and we can look at total sales by any time period. The sales figures are valid no matter how we slice the data.

While the dimension tables are short and fat, the fact tables are generally long and skinny. They are long because they can hold the number of records represented by the product of the counts in all the dimension tables.

What is data mining?

Data mining has been defined as "the nontrivial extraction of implicit, previously unknown, and potentially useful information from data" and "the science of extracting useful information from large data sets or databases".

Data mining involves sorting through large amounts of data and picking out relevant information. It is usually used by Business intelligence organizations, and financial analysts, but is increasingly used in the sciences to extract information from the enormous data sets generated by modern experimental and observational methods.

What is oracle OLAP Solution?

Oracle OLAP solution has Oracle DB OLAP Option and Hyperion Essbase.

Oracle Database OLAP is optimal for accelerating query performance and
adding analytic content for SQL-based BI Tools on Oracle Database, whereas oracle’s Hyperion Essbase is ideal for performance management applications to be the de-facto world leader in planning, budgeting applications etc.

Both the OLAP Option and Essbase share certain core capabilities which are fast query performance, fast incremental update of data sets, rich calculation capabilities and dimensional calculation and query models.

Oracle OLAP—an option to Oracle Database 11g Enterprise Edition—is a calculation engine that supports the entire spectrum of advanced analytical applications. It extends the analytic capabilities of the Oracle database by providing new multidimensional datatypes, a multidimensional calculation engine, and a framework to build OLAP applications using SQL, PL/SQL, and Java. (Know more about oracle OLAP option)


Database vs. Data Warehouse

The primary difference betwen you application database and a data warehouse is that while the former is designed (and optimized) to record , the latter has to be designed (and optimized) to respond to analysis questions that are critical for your business. Application databases are OLTP (On-Line Transaction Processing) systems where every transation has to be recorded, and super-fast at that.This system is write-optimized, and you shouldn?¡¥t crib if your analysis query (read operation) takes a lot of time on such a system.

A Data Warehouse (DW) on the other end, is a database (yes, you are right, it?¡¥s a database) that is designed for facilitating querying and analysis. Often designed as OLAP (On-Line Analytical Processing) systems, these databases contain read-only data that can be queried and analysed far more efficiently as compared to your regular OLTP application databases. In this sense an OLAP system is designed to be read-optimized.

Creation of a DW leads to a direct increase in quality of analyses as the table structures are simpler (you keep only the needed information in simpler tables), standardized (well documented table structures), and denormalized (to reduce the linkages between tables and the corresponding complexity of queries). A DW drastically reduces the ??cost-per-analysis?¡¥ and thus permits more analysis per FTE. Having a well-designed DW is the foundation successful BI/Analytics initiatives are built upon (original post).

Data Warehouse vs Data Mart

Data Warehouses and Data Marts are conceptually different in scope. However, they are built using the exact same methods and procedures.

A Data Mart is a specific, subject oriented, repository of data designed to answer specific questions for a specific set of users. So an organization could have multiple data marts serving the needs of marketing, sales, operations, collections, etc. A data mart usually is organized as one dimensional model as a star-schema (OLAP cube) made of a fact table and multiple dimension tables.

In contrast, a Data Warehouse (DW) is a single organizational repository of enterprise wide data across many or all subject areas. The Data Warehouse is the authoritative repository of all the fact and dimension data (that is also available in the data marts) at an atomic level.

A data warehouse (or mart) is way of storing data for later retrieval. This retrieval is almost always used to support decision-making in the organization. That is why many data warehouses are considered to be DSS (Decision-Support Systems). oth a data warehouse and a data mart are storage mechanisms for read-only, historical, aggregated data.

As technology improved (lower cost for more performance) and user requirements increased (faster data load cycle times and more features), data warehouses have evolved through several fundamental stages:

* Offline Operational Databases ?a Data warehouses in this initial stage are developed by simply copying the database of an operational system to an off-line server where the processing load of reporting does not impact on the operational system's performance.
* Offline Data Warehouse ?a Data warehouses in this stage of evolution are updated on a regular time cycle (usually daily, weekly or monthly) from the operational systems and the data is stored in an integrated reporting-oriented data structure
* Real Time Data Warehouse ?a Data warehouses at this stage are updated on a transaction or event basis, every time an operational system performs a transaction (e.g. an order or a delivery or a booking etc.)
* Integrated Data Warehouse ?a Data warehouses at this stage are used to generate activity or transactions that are passed back into the operational systems for use in the daily activity of the organization.

The term data warehouse architecture is primarily used today to describe the overall structure of a Business Intelligence system. Other historical terms include decision support systems (DSS), management information systems (MIS), and others (original post).

List of reference i used to compile this primer:

Introduction to OLAP
http://www.dwreview.com/OLAP/Introduction_OLAP.html

Data Warehousing Overview
http://www.dwreview.com/DW_Overview.html

Designing OLAP Solutions
http://www.dwreview.com/OLAP/OLAP_Comparison.html

OLAP architectures
http://www.olapreport.com/Architectures.htm

Designing the Star Schema Database By Craig Utley
http://www.ciobriefings.com/whitepapers/StarSchema.asp

Open Source Analytics / Open source BI by Nishith
http://opensourceanalytics.com/

Discussion of designing OLAP Cubes
http://databaseanswers.org/designing_olap_cubes.htm

Vector graphics Vs Raster graphics

Source: wikimedia.org
Vector graphics (also called geometric modeling or object-oriented graphics) is the use of geometrical primitives such as points, lines, curves, and polygons, which are all based upon mathematical equations to represent images in computer graphics.

It is used in contrast to the term raster graphics, which is the representation of images as a collection of pixels, and used as the sole graphic type for actual photographic images. To know more about vector | raster

Oracle Locator Vs Oracle Spatial

To get a quick idea of what difference between oracle locator and spatial are, i put them down here.

Features included in Oracle Locator:

* Exactly the same code base as Oracle Spatial for included functions - you won't get a performance degrade by using Oracle Locator, and features behave in the same manner in Oracle Locator and Oracle Spatial;
* Full support to store spatial data types;
* Standard SQL query access to spatial data;
* All indexing types are available - including R-Tree and partition support;
* No limits on the size and complexity of data stored;
* Most of the common spatial operations, such as "find all customers in this region" and "what shops are within 100 metres of this school", are available;
* Data compatibility between Oracle Spatial and Locator, so if you do decide to upgrade, all your data remains valid; and
* Support for multiple coordinate systems.

Oracle Spatial provides the following functions which are not included in Oracle Locator:

* Linear referenced features e.g. roads which store the distance along the road;
* Network models, such as road networks, allowing rapid calculation of shortest paths;
* Topological relationships can be stored, such as this property boundary touches this neighbouring boundary. Generally used by data producers wishing to edit high integrity spatial data;
* Storage of images such as satellite and aerial photography;
* A geocoder that translates address data to coordinate; and
* Some more complex spatial functions, mainly for analysis and data aggregation.

Find the original article here.

There a list of relevant readings from where you can go further.

Oracle Spatial - Wikipedia
Oracle Spatial Documents - Documentation
Oracle Locator and Spatial Option - Datasheet
Oracle Locator and Spatial FAQ - FAQ
Oracle XE , Locator & Geospatial Information Systems - Interview

Friday, September 28, 2007

Tech Forum & Workshop in Kuala Lumpur, Singapore & Indonesia

I have been staying in KL for the last 4 days for a tech forum and hands-on workshop hosted in KL.

This forum explains how ISVs can build their applications using oracle technology and grow their business with market leaders (Oracle & Sun). It also covers the new features, benefit and how to use technical resources available exclusively to our ISVs.

In the workshop, Oracle (My team) will assist ISVs attendees migrating their application from other databases or middleware to Oracle technology.

Here is the event agenda

Day 1:

09:00 AM Registration
09:30 AM Welcome note and agenda
09:45 AM Tech Talk By Sun (Open Solaris and Virtualization)
10:45 AM Morning Break
11:00 AM Tech Talk By Oracle ( Migration Methodology and Best Practices)
12:00 PM Lunch
01:30 PM Migration workshop
03:30 PM Afternoon Break
03:45 PM Migration workshop
06:00 OM Close

Day 2:

09:00 AM Continue Migration workshop
10:45 AM Morning Break
12:00 PM Lunch
01:30 PM Continue Migration workshop
03:30 PM Afternoon Break
03:45 PM Continue Migration workshop
06:00 OM Closing Ceremony


I reached KL two days earlier before the event open. For the first two days of my staying, I am quite busy in preparing the user environment like work stations, servers (from Sun) and network setup as this is the first time for our Malaysia partner (co-organizer) hosting this kind of event. At the last 2 days, i gave the technical talk about migration methodology and help ISVs solve their issues during the hands-on migration session.

In my speech, i covered the migration methodology, approaches and things they need to consider for the DB migration , J2EE & Application Server (AS) migration and Oracle Forms migration. Some migration tools from oracle have been introduced to our ISVs, with which migration can be automated & simplified and duration of migration can be shorten. Demonstration of how to use those tools is also given after the speech.

Here is the material for the migration, you can dig out more from the links i am giving in the ppt.

There are actually some photos taken by our partner on that day and I am in the midst of geting them and upload here. So, keep tuned for the photos.

Migration workshop in Kuala Lumpur

Migration workshop in Kuala Lumpur

Tech Forum in Singapore

Sun Talk about Virtualization in Singapore

Tech Forum in Indonesia

Talk about DB Migration in Indonesia

Talk about DB Migration in Indonesia

Thursday, September 27, 2007

Real life architectures

Want to know how the architectures in real world which scale to millions of users looks like? Visit highscalability.com, You can find these information for google, amazon, youtube and flickr.

Google Architecture
http://highscalability.com/google-architecture

Amazon Architecture
http://highscalability.com/amazon-architecture

YouTube Architecture
http://highscalability.com/youtube-architecture


Flickr Architecture
http://highscalability.com/flickr-architecture

There are more real life architectures and you can find it here.

Highscalability.com is really a great work and i had added it to my rss feed list.

Thursday, August 2, 2007

A portfolio of Oracle content management product

Oracle content management product portfolio includes Oracle Universal Content Management, Oracle Universal Records Management, Oracle Imaging and Process Management, Oracle Information Rights Management and Oracle Content Database.

- Oracle Universal Content Management: provides a unified application for delivering document management, web content management, digital asset management, and retention management on a single platform. more ...

- Oracle Universal Records Management: provides the most comprehensive DoD 5015.2 compliant records management system, enabling customers to apply records and retention management policies and practices to content in remote multiple repositories and applications including file systems, content management systems, and email archives as well as addressing the management of physical records in the same system. more ...

- Oracle Imaging and Process Management: provides image scanning and business process management capabilities for enterprise applications such as accounts payable and time and expense activities and includes support for PeopleSoft, JD Edwards, and Oracle eBusiness Suite. Its complete imaging management--from creation to archiving -includes certified integrations with Oracle Applications (including the Oracle E-Business Suite, JD Edwards, and PeopleSoft Enterprise) as well as other third-party business applications. more ...


- Oracle Information Rights Management: provides policy-based security and retention management on documents when they leave a repository. It uses encryption to "seal" documents to secure and track them everywhere they're stored and used-enablin organizations to define fine-grained policies for viewing, editing, printing, and copying sensitive information, ensuring that only authorized users have access to secured content. And, when the content or user is no longer valid, rights can be easily revoked. more ...

- Oracle Content Database Suite: an enterprise content infrastructure, that is highly scalable for global deployments and is based on the industry leading and highly scalable Oracle database. Supports file server and archive consolidation as well as a content repository for enabling and building content-centric applications. more ...

First three products are bundled together and known as Enterprise Content Management Suite.

There is a short story about what oracle did recently in the cm development, check it out if you are interested.

´