Li Hai Tao's Blog

Oracle Universal Content Management (UCM) aka Stellent

2011-06-08T19:14:00.000-07:00

Original post from here

Oracle Universal Content management or UCM is what was erstwhile Stellent Content Server. It is an enterprise wide content management, revisioning and controlling system.

In layman's terms the idea is to organize the documents of your company on a central server maintaining informations (metadata) about the documents and tracking the progress of the documents through revisioning and version control. Users would be able to upload (check in) documents based on their permission levels (through security groups and accounts) and before the document is released for everyone to view and download (check out), an approval heirarchy (workflow) can be set.

This simple concept brings in an amazing level of order in the chaos that can be in an enterprise's document management requirements.

Other than simply checking in and checking out documents, the list of functionalities that come with UCM are manyfold. Your companies website can be hosted through UCM where users simply check in their contribution as word documents and instantly the content appears on the website in a predefined template. Project documents of your company are handled by Collaboration Projects component of UCM where "on the fly" access and document management can be handled.

This brings me to the important part of components in UCM (stellent). UCM is an extensible content management system which means that through installing custom components, functionalities can be added or modified. Notable components are folders, dynamic convertor, pdf convertor, threaded discussions, collaboration projects, extranetLook, folios etc and many others.

Architecture of Stellent or Universal Content Management

The Main players in the working of UCM are

Web Browser

A user interacts with the content server through a web browser. Even though through the Desktop Integration Component (DIS) windows explorer can be used as an interface, however the simple web browser is the default interface.

Web Server

Obviously, a web server will be handling the request sent by the web browser to the content server. Microsoft IIS, Apache, Oracle iAS can all be used as a web server of choice. Installing with IIS is actually the easiest.

Content Server

Content Server is the core service which provides all the functionalities. At it's core Content Server consists of Java Applications that require a Java Runtime environment installed on your server.

Vault and Web Layout

When users check-in content, they are stored in a folder on the server called vault in their "native format". However, if you have installed the dynamic convertor a web layout directory stores the html conversions of your native files.

Search Index and Search Engine

Searching is the most useful and the most necessary feature of the content server. Users can search for content by querying on the metadata or use a Full Text Searching provided you have dynamic convertor installed.

Oracle Database

Oracle UCM runs on DB2, Microsoft SQL server in addition to Oracle. When installing UCM you will need to create a user for Content Server to use to connect to the database.

So these were the principal components of Oracle UCM. An image is attached which shows gives a graphical representation of the architecture.

To get to know more, read following documents on Club-Oracle

•Oracle UCM or stellent Beginner's Documents and tutorials
•Installing Stellent or Oracle Universal Content Management
•Steps to install Site Studio and other components
•How to create a Dependent Choice list in Oracle UCM (Stellent)

Hyperion Reporting and Analysis Architecture

2011-05-25T00:48:00.000-07:00

Various components of Hyperion (source):

Workspace is a common window to view the contents of all Hyperion components.
Hyperion Reporting and Analysis:

One zero-footprint Web-based thin client provides users with access to content:
● Financial reporting for scheduled or on-demand highly formatted financial and operational reporting from most data sources including Hyperion Planning – System 9 and Hyperion Financial Management – System 9
● Interactive reporting for ad hoc relational queries, self-service reporting and dashboards against ODBC data sources
● SQR Production reporting for high volume enterprise-wide production reporting.
● Web analysis for interactive ad hoc analysis, presentation, and reporting of multidimensional data.

Hyperion Reporting and Analysis Architecture

Client: The client tools consist of

Workspace: It is a DHTML Zero footprint web client and provide the user interface for viewing and interacting with the reports created using Authoring studios.
Authoring Studios: These are the client interfaces to create the reports and includes-

(a) Hyperion Interactive Reporting Studio: Windows client where you can connect to different data sources including the flat files and build very interactive presentation reports like reports in simple tabular format, pivot reports, graphs and charts with drill anywhere feature which means that you don’t have to define the hierarchy or drill path and slicing and dicing and Dashboards with many features like hyperlinks to the details reports and embedded browser which can be used to view any other web application to open within the Dashboards.

(b) Hyperion Financial Reporting Studio: Windows client where you can connect to the multidimentional data sources and create highly formatted financial reports by simply dragging and dropping rows and columns and defining page breaks.

(c) Hyperion SQR Reporting Studio: Windows client where you can connect to wide range of data sources and produce high volume pixel perfect operational reports and can be scheduled.

(d) Hyperion Web Analysis: Java applet where you can connect to different data sources using JDBC and build interactive reports and dashboards.
Smart view for office: This is a tight integration with Microsoft Office tools where ou can do analysis like drill downs, keep only and remove only options, POV manager, data refresh, copying data cells and pasting to MS Word and Powerpoint which automatically gets refreshed if the data changes in the source etc. There is one more component in smart wiew which is Hyperion Visual Explorer(HVE), where again you can view the data in presentable interactive graphs and charts.

Application Layer: It consists of two parts :

1. Web Tier: It consists of two parts (a) Web server- to send and receives content from the web clients. (b) Application server- it is a J2EE application server.
Web server and application server are connected using an HTTP connector.
The web Tier hosts the web applications like workspace, web analysis, interactive , SQR and financial reporting applications.

2. Services Tier: It contains services and servers that controls the functionality of the web applications and clients. Core services handles repository information, authorization, session information, documents publication.

More to read:

http://www.youtube.com/watch?v=j5REDY8cpiM&NR=1
http://download.oracle.com/docs/cd/E12032_01/doc/nav/portal_1.htm
http://businessintelligence.ittoolbox.com/groups/technical-functional/hyperion-admin-l/ir-92-hyperion-biservice-is-not-accessible-1543368
http://businessintelligence.ittoolbox.com/groups/technical-functional/brio-l/hyperion-system-93-adding-additional-bi-service-memory-exhausted-4039255
http://businessintelligence.ittoolbox.com/groups/technical-functional/brio-l/active-x-client-hyperion-931-out-of-memory-on-db2-2288146
http://businessintelligence.ittoolbox.com/groups/technical-functional/brio-l/hyperion-out-of-memory-error-2099199
http://businessintelligence.ittoolbox.com/groups/technical-functional/hyperion-bi-l/hyperion-designer-85-out-of-memory-for-large-queries-1332854
http://essbase.ru/archives/wiki/obiee-11gr1-architecture-and-use-of-weblogic-server
https://www.packtpub.com/toc/business-analysts-guide-oracle-hyperion-interactive-reporting-11-table-contents

Portlets & Portal

2011-05-23T00:44:00.000-07:00

Introduction

JSR 168 Vs JSR 286

Portlets Vs Servlets

Portlet Lifecycle

Penetration Test and Scanner Benchmark

2011-05-19T03:30:00.000-07:00

Introduction (source from Corsaire)

Penetration testing is an often confused term. Through this guide Corsaire, a world leader in information security, provides a broad overview of what it means, why you would want it, and how to get the most out of the process.

What is a penetration test?
Why conduct penetration testing?
What can be tested?
What should be tested?
What do you get for the money?
What to do to ensure the project is a success
What is a penetration test?
Much of the confusion surrounding penetration testing stems from the fact it is a relatively recent and rapidly evolving field. Additionally, many organisations will have their own internal terminology (one man's penetration test is another's vulnerability audit or technical risk assessment).

At its simplest, a penetration-test (actually, we prefer the term security assessment) is the process of actively evaluating your information security measures. Note the emphasis on 'active' assessment; the information systems will be tested to find any security issues, as opposed to a solely theoretical or paper-based audit.

The results of the assessment will then be documented in a report, which should be presented at a debriefing session, where questions can be answered and corrective strategies can be freely discussed.

Why conduct a penetration test?
From a business perspective, penetration testing helps safeguard your organisation against failure, through:

Preventing financial loss through fraud (hackers, extortionists and disgruntled employees) or through lost revenue due to unreliable business systems and processes.
Proving due diligence and compliance to your industry regulators, customers and shareholders. Non-compliance can result in your organisation losing business, receiving heavy fines, gathering bad PR or ultimately failing. At a personal level it can also mean the loss of your job, prosecution and sometimes even imprisonment.
Protecting your brand by avoiding loss of consumer confidence and business reputation.
From an operational perspective, penetration testing helps shape information security strategy through:

Identifying vulnerabilities and quantifying their impact and likelihood so that they can be managed proactively; budget can be allocated and corrective measures implemented.
What can be tested?
All parts of the way that your organisation captures, stores and processes information can be assessed; the systems that the information is stored in, the transmission channels that transport it, and the processes and personnel that manage it. Examples of areas that are commonly tested are:

Off-the-shelf products (operating systems, applications, databases, networking equipment etc.)
Bespoke development (dynamic web sites, in-house applications etc.)
Telephony (war-dialling, remote access etc.)
Wireless (WIFI, Bluetooth, IR, GSM, RFID etc.)
Personnel (screening process, social engineering etc.)
Physical (access controls, dumpster diving etc.)
What should be tested?
Ideally, your organisation should have already conducted a risk assessment, so will be aware of the main threats (such as communications failure, e-commerce failure, loss of confidential information etc.), and can now use a security assessment to identify any vulnerabilities that are related to these threats. If you haven't conducted a risk assessment, then it is common to start with the areas of greatest exposure, such as the public facing systems; web sites, email gateways, remote access platforms etc.

Sometimes the 'what' of the process may be dictated by the standards that your organisation is required to comply with. For example, a credit-card handling standard (like PCI) may require that all the components that store or process card-holder data are assessed.

What do you get for the money?
While a great deal of technical effort is applied during the testing and analysis, the real value of a penetration test is in the report and debriefing that you receive at the end. If they are not clear and easy to understand, then the whole exercise is of little worth.

Ideally the report and debriefing should be broken into sections that are specifically targeted at their intended audience. Executives need the business risks and possible solutions clearly described in layman's terms, managers need a broad overview of the situation without getting lost in detail, and technical personnel need a list of vulnerabilities to address, with recommended solutions.

What to do to ensure the project is a success
Defining the scope

The scope should be clearly defined, not only in the context of the components to be (or not to be) assessed and the constraints under which testing should be conducted, but also the business and technical objectives. For example penetration testing may be focussed purely on a single application on a single server, or may be more far reaching; including all hosts attached to a particular network.

Choosing a security partner

Another critical step to ensure that your project is a success is in choosing which supplier to use.

As an absolute fundamental when choosing a security partner, first eliminate the supplier who provided the systems that will be tested. To use them will create a conflict of interest (will they really tell you that they deployed the systems insecurely, or quietly ignore some issues).

Detailed below are some questions that you might want to ask your potential security partner:

Is security assessment their core business?
How long have they been providing security assessment services?
Do they offer a range of services that can be tailored to your specific needs?
Are they vendor independent (do they have NDAs with vendors that prevent them passing information to you)?
Do they perform their own research, or are they dependent on out-of-date information that is placed in the public domain by others?
What are their consultant's credentials?
How experienced are the proposed testing team (how long have they been testing, and what is their background and age)?
Do they hold professional certifications, such as PCI, CISSP, CISA, and CHECK?
Are they recognised contributors within the security industry (white papers, advisories, public speakers etc)?
Are the CVs available for the team that will be working on your project?
How would the supplier approach the project?
Do they have a standardised methodology that meets and exceeds the common ones, such as OSSTMM, CHECK and OWASP?
Can you get access to a sample report to assess the output (is it something you could give to your executives; do they communicate the business issues in a non-technical manner)?
What is their policy on confidentiality?
Do they outsource or use contractors?
Are references available from satisfied customers in the same industry sector?
Is there a legal agreement that will protect you from negligence on behalf of the supplier?
Does the supplier maintain sufficient insurance cover to protect your organisation?
Standards compliance
There are a number of good standards and guidelines in relation to information security in general, for penetration tests in particular, and for the storage of certain types of data. Any provider chosen should at least have a working knowledge of these standards and would ideally be exceeding their recommendations.

Notable organisations and standards include:

PCI
The Payment Card Industry (PCI) Data Security Requirements were established in December 2004, and apply to all Members, merchants, and service providers that store, process or transmit cardholder data. As well as a requirement to comply with this standard, there is a requirement to independently prove verification.

ISACA
ISACA was established in 1967 and has become a pace-setting global organization for information governance, control, security and audit professionals. Its IS Auditing and IS Control standards are followed by practitioners worldwide and its research pinpoints professional issues challenging its constituents. CISA, the Certified Information Systems Auditor is ISACA's cornerstone certification. Since 1978, the CISA exam has measured excellence in the area of IS auditing, control and security and has grown to be globally recognized and adopted worldwide as a symbol of achievement.

CHECK
The CESG IT Health Check scheme was instigated to ensure that sensitive government networks and those constituting the GSI (Government Secure Intranet) and CNI (Critical National Infrastructure) were secured and tested to a consistent high level. The methodology aims to identify known vulnerabilities in IT systems and networks which may compromise the confidentiality, integrity or availability of information held on that IT system. In the absence of other standards, CHECK has become the de-facto standard for penetration testing in the UK. This is mainly on account of its rigorous certification process. Whilst good it only concentrates on infrastructure testing and not application. However, open source methodologies such as the following are providing viable and comprehensive alternatives, without UK Government association. It must also be noted that CHECK consultants are only required when the assessment is for HMG or related parties, and meets the requirements above. If you want a CHECK test you will need to surrender your penetration testing results to CESG.

OSSTMM
The aim of The Open Source Security Testing Methodology Manual (OSSTMM) is to set forth a standard for Internet security testing. It is intended to form a comprehensive baseline for testing that, if followed, ensures a thorough and comprehensive penetration test has been undertaken. This should enable a client to be certain of the level of technical assessment independently of other organisation concerns, such as the corporate profile of the penetration-testing provider.

OWASP
The Open Web Application Security Project (OWASP) is an Open Source community project developing software tools and knowledge based documentation that helps people secure web applications and web services. It is an open source reference point for system architects, developers, vendors, consumers and security professionals involved in designing, developing, deploying and testing the security of web applications and Web Services.

The key areas of relevance are the forthcoming Guide to Testing Security of Web Applications and Web Services and the testing tools under the development projects. The Guide to Building Secure Web Applications not only covers design principals, but also is a useful document for setting out criteria by which to assess vendors and test systems.

Scanner Benchmark

Check it out

Some userful tools:

OWASP Live CD

WebScarab

Caching and Frameworks

2011-05-17T21:01:00.000-07:00

Intro to caching and caching frameworks.

What Are the Trade-Offs Involved in Caching?

Caching is always based on compromise; a trade-off between performance, scalability, and accuracy using the various resources available. Ease of configuration is an important secondary consideration. We should consider how we balance these factors to achieve the best performance possible.

Application performance depends on efficient data distribution. It's crucial to ensure fast data access for maximum application performance. It pays to build a cache and avoid unnecessary round-trips to the datastore. By reducing traffic between the different layers of an application, you can substantially diminish the size and cost of the installation and greatly enhance the system's responsiveness.

Hibernate Caching

Distributed and Replicated Caching

Using JCache

Open Source Cache Solutions in Java

SecureCI

2011-05-12T03:30:00.000-07:00

A VM containing a turn-key solution for continuous integration with source code control, build management, automated testing, security analysis, defect tracking, and project management, all using open source tools.

Check it out

Open Source Application Server Comparison

2011-05-12T03:23:00.000-07:00

Things to consider when choosing open source application servers

•Load balancing: random; minimum load; round-robin; weighted round-robin; performance-based; load-based; dynamic algorithm based; dynamic registration.
•Clustering & HA. Additionally: distributed transaction management; in-memory replication of session state information; no single point of failure.
•Connection pooling.
•Caching. JNDI caching. Distributed caching with synchronization.
•Thread pooling.
•Configurable user Quality of Service.
•Analysis tools.
•Low system/memory requirements.
•Optimized subsystems (RMI, JMS, JDBC drivers, JSP tags & cacheable page fragments).
•Optimistic transaction support.

Wikipedia Comparision

Openlogic (Summary | Full)

Web Application and App Server Performance Tunning Tips

2011-05-12T03:18:00.000-07:00

Source content is from Java Performance Tuning

The following pages have their detailed tips extracted below

•J2EE Application server performance
•Tuning IBM's WebSphere product
•WebSphere V3 Performance Tuning Guide
•Weblogic tuning (generally applicable Java tips extracted)
•Overview of common application servers
•Web application scalability.
•J2EE Application servers
•Load Balancing Web Applications
•J2EE clustering
•"EJB2 clustering with application servers"
•Choosing an application server
•Choosing a J2EE application server, emphasizing the importance of performance issues
•Implementing clustering on a J2EE web server (JBoss+Jetty)
•Tuning tips intended for Sun's "Web Server" product, but actually generally applicable.
•Various tips
•iPlanet Web Server guide to servlets, with a section at the end on "Maximizing Servlet Performance".
•Sun community chat on iPlanet
•Article on high availability architecture

--------------------------------------------------------------------------------

The following detailed tips have been extracted from the raw tips page

J2EE Application server performance (Page last updated April 2001, Added 2001-04-20, Author Misha Davidson, Publisher Java Developers Journal). Tips:
•Good performance has sub-second latency (response time) and hundreds of (e-commerce) transactions per second.
•Avoid n-way database joins: every join has a multiplicative effect on the amount of work the database has to do. The performance degradation may not be noticeable until large datasets are involved.
•Avoid bringing back thousands of rows of data: this can use a disproportionate amount of resources.
•Cache data when reuse is likely.
•Avoid unnecessary object creation.
•Minimize the use of synchronization.
•Avoid using the SingleThreadModel interface for servlets: write thread-safe code instead.
•ServletRequest.getRemoteHost() is very inefficient, and can take seconds to complete the reverse DNS lookup it performs.
•OutputStream can be faster than PrintWriter. JSPs are only generally slower than servlets when returning binary data, since JSPs always use a PrintWriter, whereas servlets can take advantage of a faster OutputStream.
•Excessive use of custom tags may create unnecessary processing overhead.
•Using multiple levels of BodyTags combined with iteration will likely slow down the processing of the page significantly.
•Use optimistic transactions: write to the database while checking that new data is not be overwritten by using WHERE clauses containing the old data. However note that optimistic transactions can lead to worse performance if many transactions fail.
•Use lazy-loading of dependent objects.
•For read-only queries involving large amounts of data, avoid EJB objects and use JavaBeans as an intermediary to access manipulate and store the data for JSP access.
•Use stateless session EJBs to cache and manage infrequently changed data. Update the EJB occasionally.
•Use a dedicated session bean to perform and cache all JNDI lookups in a minimum number of requests.
•Minimize interprocess communication.
•Use clustering (multiple servers) to increase scalability.

Tuning IBM's WebSphere product. White paper: "Methodology for Production Performance Tuning". Only non-product specific Java tips have been extracted here. (Page last updated September 2000, Added 2001-01-19, Author Gennaro (Jerry) Cuomo, Publisher IBM). Tips:
•A size restricted queue (closed queue) allows system resources to be more tightly managed than an open queue.
•The network provides a front-end queue. A server should be configured to use the network queue as its bottleneck, i.e. only accept a request from the network when there are sufficient resources to process the request. This reduces the load on an app server. However, sufficient requests should be accepted to ensure that the app server is working at maximum capacity, i.e. try not to let a component sit idle while there are still requests that can be accepted even if other components are fully worked.
•Try to balance the workload of the various components.
•[Paper shows a nice throughput curve giving recommended scaling behavior for an server]
•The desirable target bottleneck is the CPU, i.e. a server should be tuned until the CPU is the remaining bottleneck. Adding CPUs is a simple remedy to this.
•Use connection pools and cached prepared statements for database access.
•Object memory management is particularly important for server applications. Typically garbage collection could take between 5% and 20% of the server execution time. Garbage collection statistics provide a useful monitor to determine the server's "health". Use the verbosegc flag to collect basic GC statistics.
•GC statistcs to monitor are: total time spent in GC (target less than 15% of execution time); average time per GC; average memory collected per GC; average objects collected per GC.
•For long lived server processes it is particularly important to eliminate memory leaks (references retained to objects and never released).
•Use -ms and -mx to tune the JVM heap. Bigger means more space but GC takes longer. Use the GC statistics to determine the optimal setting, i.e the setting which provides the minimum average overhead from GC.
•The ability to reload classes is typically achieved by testing a filesystem timestamp. This check should be done at set intermediate periods, and not on every request as the filesystem check is an expensive operation.

WebSphere V3 Performance Tuning Guide (Page last updated March 2000, Added 2001-01-19, Authors Ken Ueno, Tom Alcott, Jeff Carlson, Andrew Dunshea, Hajo Kitzhöfer, Yuko Hayakawa, Frank Mogus, Colin D. Wordsworth, Publisher IBM). Tips:
•[The Red book lists and discusses tuning parameters available to Websphere]
•Run an application server and any database servers on separate server machines.
•JVM heap size: -mx, -ms [-Xmx, -Xms]. As a starting point for a server based on a single JVM, consider setting the maximum heap size to 1/4 the total physical memory on the server and setting the minimum to 1/2 of the maximum heap. Sun recommends that ms be set to somewhere between 1/10 and 1/4 of the mx setting. They do not recommend setting ms and mx to be the same. Bigger is not always better for heap size. In general increasing the size of the Java heap improves throughput to the point where the heap no longer resides in physical memory. Once the heap begins swapping to disk, Java performance drastically suffers. Therefore, the mx heap setting should be set small enough to contain the heap within physical memory. Also, large heaps can take several seconds to fill up, so garbage collection occurs less frequently which means that pause times due to GC will increase. Use verbosegc to help determine the optimum size that minimizes overall GC.
•In some cases turning off asynchronous garbage collection ("-noasyncgc", not always available to all JVMs) can improve performance.
•Setting the JVM stack and native thread stack size (-oss and -ss) too large (e.g. greater than 2MB) can significantly degrade performance.
•When security is enabled (e.g. SSL, password authentication, security contexts and access lists, encryption, etc) performance is degraded by significant amounts.
•One of the most time-consuming procedures of a database application is establishing a connection to the database. Use connection pooling to minimize this overhead.

Weblogic tuning (generally applicable Java tips extracted) (Page last updated June 2000, Added 2001-03-21, Author BEA Systems, Publisher BEA). Tips:
•Response time is affected by: contention and wait times, particularly for shared resources; and software and hardware component performance, i.e. the amount of time that resources are needed.
•A well-designed application can increase performance by simply adding more resources (for instance, an extra server).
•Use clustered or multi-processing machines; use a JIT-enabled JVM; use Java 2 rather than JDK 1.1;
•Use -noclassgc. Use the maximum possible heap size that also is small enough to avoid the JVM from swapping (e.g. 80% of RAM left over after other required processes). Consider starting with minimum initial heap size so that the garbage collector doesn't suddenly encounter a full heap with lots of garbage. Benchmarkers sometimes like to set the heap as high as possible to completely avoid GC for the duration of the benchmark.
•Distributing the application over several server JVMs means that GC impact will be spread in time, i.e. the various JVMs will most likely GC at different times from each.
•On Java 1.1 the most effective heap size is that which limits the longest GC incurred pause to the longest acceptable pause in processing time. This will typically require a reduction in the maximum heap size.
•Too many threads causes too much context switching. Too few threads may underutilize the system. If n=number of threads, k=number of CPUs, then: (n < k) results in an under utilized CPU; (n == k) is theoretically ideal, but each CPU will probably be under utilized; (n > k) by a "moderate amount of threads" is practically ideal; (n > k) by "many threads" can lead to significant performance degradation from context switching. Blocked threads count for less in the previous formulae.
•Symptoms of too few threads: CPU is waiting to do work, but there is work that could be done; Can not get 100% CPU; All threads are blocked [on i/o] and runnable when you do an execution snapshot.
•Symptoms of too many threads: An execution snapshot shows that there is a lot of context switching going on in your JVM; Your performance increases as you decrease the number of threads.
•If many client connections are dropped or refused, the TCP listen queue may be too short.
•Try to avoid excessive cycling (creation/deletion or activation/passivation) of beans.

Overview of common application servers. I've extracted the performance related features (Page last updated October 2001, Added 2001-10-22, Author Pieter Van Gorp, Publisher Van Gorp). Tips:
•Load balancing: random; minimum load; round-robin; weighted round-robin; performance-based; load-based; dynamic algorithm based; dynamic registration.
•Clustering. Additionally: distributed transaction management; in-memory replication of session state information; no single point of failure.
•Connection pooling.
•Caching. JNDI caching. Distributed caching with synchronization.
•Thread pooling.
•Configurable user Quality of Service.
•Analysis tools.
•Low system/memory requirements.
•Optimized subsystems (RMI, JMS, JDBC drivers, JSP tags & cacheable page fragments).
•Optimistic transaction support.

Web application scalability. (Page last updated June 2000, Added 2001-05-21, Author Billie Shea, Publisher STQE Magazine). Tips:
•Web application scalability is the ability to sustain the required number of simultaneous users and/or transactions, while maintaining adequate response times to end users.
•The first solution built with new skills and new technologies will always have room for improvement.
•Avoid deploying an application server that will cause embarrassment, or that could weaken customer confidence and business reputation [because of bad response times or lack of calability].
•Consider application performance throughout each phase of development and into production.
•Performance testing must be an integral part of designing, building, and maintaining Web applications.
•There appears to be a strong correlation between the use of performance testing tools and the likelihood that a site would scale as required.
•Automated performance tests must be planned for and iteratively implemented to identify and remove bottlenecks.
•Validate the architecture: decide on the maximum scaling requirements and then performance test to validate the necessary performance is achievable. This testing should be done on the prototype, before the application is built.
•Have a clear understanding of how easily your configurations of Web, application, and/or database servers can be expanded.
•Factor in load-balancing software and/or hardware in order to efficiently route requests to the least busy resource.
•Consider the effects security will have on performance: adding a security layer to transactions will impact response times. Dedicate specific server(s) to handle secure transactions.
•Select performance benchmarks and use them to quantify the scalability and determine performance targets and future performance improvements or degradations. Include all user types such as "information-gathering" visitors or "transaction" visitors in your benchmarks.
•Perform "Performance Regression Testing": continuously re-test and measure against the established benchmark tests to ensure that application performance hasn?t been degraded because of the changes you?ve made.
•Performance testing must continue even after the application is deployed. For applications expected to perform 24/7 inconsequential issues like database logging can degrade performance. Continuous monitoring is key to spotting even the slightest abnormality: set performance capacity thresholds and monitor them.
•When application transaction volumes reach 40% of maximum expected volumes, it is time to start executing plans to expand the system

J2EE Application servers (Page last updated April 2001, Added 2001-04-20, Authors Christopher G. Chelliah and Sudhakar Ramakrishnan, Publisher Java Developers Journal). Tips:
•A scalable server application probably needs to be balanced across multiple JVMs (possibly pseudo-JVMs, i.e. multiple logical JVMs running in the same process).
•Performance of an application server hinges on caching, load balancing, fault tolerance, and clustering.
•Application server caching should include web-page caches and data access caches. Other caches include caching servers which "guard" the application server, intercepting requests and either returning those that do not need to go to the server, or rejecting or delaying those that may overload the app server.
•Application servers should use connection pooling and database caching to minimize connection overheads and round-trips.
•Load balancing mechanisms include: round-robin DNS (alternating different IP-addresses assigned to a server name); and re-routing mechanisms to distribute requests across multiple servers. By maintaining multiple re-routing servers and a client connection mechanism that automatically checks for an available re-routing server, fault tolerance is added.
•Using one thread per user can become a bottleneck if there are a large number of concurrent users.
•Distributed components should consider the proximity of components to their data (i.e., avoid network round-trips) and how to distribute any resource bottlenecks (i.e., CPU, memory, I/O) across the different nodes.

Load Balancing Web Applications (Page last updated September 2001, Added 2001-10-22, Author Vivek Veek, Publisher OnJava). Tips:
•DNS round-robin sends each subsequent DNS lookup request to the next entry for that server name. This provides a simple machine-level load-balancing mechanism, but is only appropriate for session independent or shared-session servers.
•DNS round-robin has no server load measuring mechanisms, so requests can still go to overloaded servers, i.e. the load balancing can be very unbalanced.
•Hardware load-balancers solve many of the problems of DNS round-robin, but introduce a single point of failure.
•A web server proxy can also provide load-balancing by redirecting requests to multiple backend webservers.

J2EE clustering (Page last updated August 2001, Added 2001-08-20, Author Abraham Kang, Publisher JavaWorld). Tips:
•Consider cluster-related and load balancing programming issues from the beginning of the development process.
•Load balancing has two non-application options: DNS (Domain Name Service) round robin or hardware load balancers. [Article discusses the pros and cons].
•To support distributed sessions, make sure: all session referenced objects are serializable; store session state changes in a central repository.
•Try to keep multiple copies of objects to a minimum.

"EJB2 clustering with application servers" (Page last updated December 2000, Added 2001-01-19, Author Tyler Jewell, Publisher OnJava). Tips:
•[Article discusses multiple independent ways to load balance EJBs]

Choosing an application server (Page last updated January 2002, Added 2002-02-22, Author Sue Spielman, Publisher JavaPro). Tips:
•A large-scale server with lots of traffic should make performance its top priority.
•Performance factors to consider include: connection pooling; types of JDBC drivers; caching features, and their configurability; CMP support.
•Inability to scale with reliable performance means lost customers.
•Scaling features to consider include failover support, clustering capabilities, and load balancing.

Choosing a J2EE application server, emphasizing the importance of performance issues (Page last updated February 2001, Added 2001-02-21, Author Steve Franklin, Publisher DevX). Tips:
•Application server performance is affected by: the JDK version; connection pooling availability; JDBC version and optimized driver support; caching support; transactional efficiency; EJB component pooling mechanisms; efficiency of webserver-appserver connection; efficiency of persistence mechanisms.
•Your application server needs to be load tested with scaling, to determine suitability.
•Always validate the performance of the app server on the target hardware with peak expected user numbers.
•Decide on what is acceptable downtime for your application, and ensure the app server can deliver the required robustness. High availability may require: transparent fail-over; clustering; load balancing; efficient connection pooling; caching; duplicated servers; scalable CPU support.

Implementing clustering on a J2EE web server (JBoss+Jetty) (Page last updated September 2001, Added 2001-10-22, Author Bill Burke, Publisher OnJava). Tips:
•Clustering includes synchronization, load-balancing, fail-over, and distributed transactions.
•[article discusses implementing clustering in an environment where clustering was not previously present].
•The different EJB commit options affect database traffic and performance. Option 'A' (read-only local caching) has the smallest overhead.
•Hardware load balancers are a simple and fast solution to distributing HTTP requests to clustered servers.

Tuning tips intended for Sun's "Web Server" product, but actually generally applicable. (Page last updated 1999, Added 2000-10-23, Author ? - a Sun document, Publisher Aikido). Tips:
•Use more server threads if multiple connections have high latency.
•Use keep-alive sockets for higher throughput.
•Increase server listen queues for high load or high latency servers.
•Avoid or reduce logging.
•Buffer logging output: use less than one real output per log.
•Avoid reverse DNS lookups.
•Write time stamps rather than formatted date-times.
•Separate paging and application files.
•A high VM heap size may result in paging, but could avoid some garbage collections.
•Occasional very long GCs makes the VM hang for that time, leading to variability in service quality.
•Doing GC fairly often and avoiding paging is more efficient.
•Security checks consume CPU resources. You will get better performance if you can turn security checking off.

Various tips. For web servers? (Page last updated 2000, Added 2000-10-23, Author ?, Publisher ?). Tips:
•Test multiple VMs.
•Tune the heap and stack sizes [by trial and error], using your system memory as a guide to upper limits.
•Keep the system file cache large. [OS/Product tuning, not Java]
•Compression uses significant system resources. Don't use it on a server unless necessary.
•Monitor thread utilization. Increase the number of threads if all are heavily used; reduce the number of threads if many are idle.
•Empirically test for the optimal number of database connections.

iPlanet Web Server guide to servlets, with a section at the end on "Maximizing Servlet Performance". (Page last updated July 2000, Added 2001-02-21, Author ?, Publisher Sun). Tips:

•Try to optimize the servlet loading mechanism, e.g. by listing the servlet first in loading configurations.
•Tune the heap size.
•Keep the classpath short.

Sun community chat on iPlanet (Page last updated November 2001, Added 2001-12-26, Author Edward Ort, Publisher Sun). Tips:

•Optimal result caching (caching pages which have been generated) needs tuning, especially the timeout setting. Make sure the timeout is not too short.

Article on high availability architecture. If the system isn't up when you need it, its not performing. (Page last updated November 1998, Added 2000-10-23, Author Sam Wong, Publisher Sun). Tips:

•Eliminate all potential single-points-of-failure, basically with redundancy and automatic fail-over.
•Consider using the redundant components to improve performance, with a component failure causing decreased performance rather system failure.

Where are the differences between JVMs

2011-05-10T20:11:00.000-07:00

•JRockit
•IBM JVM
•SUN JVM
•Open JDK
•Blackdown
•Kaffe

JVM implementations can differ in the way they implement JIT compiling, optimizations, garbage collection, platforms supported, version of Java supported, etc. They all must meet set of features and behaviors so that it will execute Java bytecodes correctly.

The major difference tends to be in licensing. Other non-technical differences tend to be in free/paid support options, integration with other technologies (usually J2EE servers), and access to source code.

Note, While a J2EE server runs on the JVM, some servers have integrated tools for monitoring, analyzing, and tweaking JVM performance.

As far as technical differences, those have grown less siginificant over the years. Once upon a time, the IBM and JRockit JVM's had far supierior performance to the reference Sun implementation. This was due to significant differences in the types of runtime optiizations, differences in garbage collection, and differences in native-code (and how much native code various classes uses). These performance differences aren't as significant anymore.

Some JVM's also include or integrate with diagnostics and monitoring tools. JRockit includes a set of tools for monitoring your JVM performance. Sun provides various JMX-based tools with overlapping features to do the same. IBM Websphere once upon a time included a similar set of tools for their whole J2EE application server, not sure if they still do.

Some of the open source JVM's tend to have a little slower performance because they have been redeveloped from the ground up. As such, they've got a bit more catching up to do. Some say Blackdown was significantly slower than the Sun JVM and was also a bit behind of supported versions of Java.

Why JRockit is More Recommended for Production Environments ?

- It Has More diagnostic tools than other JDKs such as JRA
- Management console with no overhead on performance (JRMC)…which is available inside \bin directory.
- Better performance on Intel architectures than other JVMs
- Higher memory usage for better performance
- Great code Optimization Stretegy. (Can be Disabled using -Xnoopt is you dont want optimization)

Oracle recommends that you use JRockit JDK with your Oracle products because it has a Enhanced Garbage Collection Strategies, It have a JRMC tool inbuilt to Monitor the JVM Activities in the Runtime with very less Burden on the Server/JVM.

There are some tools allowed you to know what is the JVM doing at runtime

jdb – the debugger
jps – the process status tool, which displays process information for current Java processes
javap – the class file disassembler
javah – the C header and stub generator, used to write native methods
extcheck – a utility which can detect JAR-file conflicts
apt – the annotation-processing tool [1]
jhat – (experimental) Java heap analysis tool
jstack – (experimental) utility which prints Java stack traces of Java threads
jstat – (experimental) Java Virtual Machine statistics monitoring tool
jstatd – (experimental) jstat daemon
jinfo – (experimental) This utility gets configuration information from a running Java process or crash dump.
jmap – (experimental) This utility outputs the memory map for Java and can print shared object memory maps or heap memory details of a given process or core dump.
idlj – the IDL-to-Java compiler. This utility generates Java bindings from a given Java IDL file.
VisualVM – visual tool integrating several command-line JDK tools and lightweight[clarification needed] performance and memory profiling capabilities
jrunscript – Java command-line script shell.

There is nice post about "Open the Black Box (JVM)" with tools from different JDK vendor.

References:

Difference between JVM implementations

Why and How Oracle JRockit?

Open the Black Box - Oracle JRockit, Sun Hotspot and IBM J9 JVM Tools

DB Index and 'LIKE-clause' Tuning

2011-04-17T08:02:00.000-07:00

Understanding Indexes

Understanding Explain Plan

Turbocharge SQL with advanced Oracle indexing

Tuning a 'LIKE-clause' by using Oracle Text or Reverse Key Indexes

“Just enough & in time” Enterprise Architecture - Oracle EA Framework

2009-10-05T02:30:00.000-07:00

“The real value of Enterprise Architecture is not in making better architectures…it’s in making better enterprise.”— Gary Doucet, Chief Architect, Government of Canada Treasury Board of Canada Secretariat GC

"In the early days of computing, technology simply automated manual processes with greater efficiency. As technology evolved, new innovations enabled new capabilities and processes in the enterprise that were driven by IT. Gradually, IT changed the business but not necessarily in alignment with the business strategy. This lack of alignment resulted in significant waste of resources and missed opportunities, and placed the organization in a competitive disadvantage in the market."

"To align the strategies of business with IT, a new approach for managing IT has been developed called Enterprise Architecture. Just as architecture provides a blueprint for constructing a building, Enterprise Architecture provides a blueprint and roadmap for aligning business strategy with IT."

"Enterprise Architecture (EA) is a method and an organizing principle that aligns functional business objectives and strategies with an IT strategy and execution plan. The Enterprise Architecture provides a guide to direct the evolution and transformation of enterprises with technology. This in turn makes IT a more strategic asset for successfully implementing a modern business strategy."

"An Enterprise Architecture typically produces deliverables such as:

· Current-state Enterprise Architecture model
· Future-State Enterprise Architecture reference model that is needed to execute on the proposed business strategy
· Gap analysis that identifies the shortfalls of the current state in terms of its ability to support the objectives and strategies of the business
· Architecture Roadmap that defines the initiatives required to migrate from the current state into the future state."

"By taking an enterprisewide perspective across business services, business processes, information, applications, and technology, an EA ensures the enterprise goals and objectives are addressed in a holistic way across all IT projects."

"To be successful, an Enterprise Architecture needs to be woven into the enterprise’s culture, not treated as a closed-scope project. The value of an EA is greatly enhanced when it is organically embedded into the lifecycle of the organization, including capital planning, project management, asset management, resource allocation, and strategy formulation."

"Enterprise Architecture is a journey not a project. It evolves over time and needs to maintain the flexibility required to adjust to changing market conditions, strategy shifts, and new innovations in technology."

"A number of EA frameworks exist in the industry with the goal of addressing the basic challenge of assessing, aligning, and organizing business objectives with technical requirements and strategies. Examples include the Zachman Framework (IBM framework from 1980s), The Open Group Architecture Framework (TOGAF), OMB Federal Enterprise Architecture (FEA), The Gartner Methodology (formerly the Meta Framework), and the DoD Architecture Framework (DoDAF)."

"In an effort to provide effective framework to help customers align their IT and business strategies, Oracle extracted aspects from several industry frameworks, including TOGAF and FEA, to build a simple yet practical and prescriptive framework called the Oracle Enterprise Architecture Framework (OEAF). The OEAF is complementary to other EA frameworks, with clear mappings to TOGAF and FEA, such that customers can use the EA framework of their choice. The intent of building the OEAF was to leverage the strengths of the different industry frameworks and marry that with Oracle’s experience in developing enterprise solutions."

"Oracle emphasizes a “just enough” and “just in time” practical approach to Enterprise Architecture, which may be used standalone or as a complement a customer’s selected EA methodology."

So, what is "just enough" & "just in time", listen to this podcast.

To know more about OEAF, check out the Oracle EA whitepaper (http://www.oracle.com/technology/architect/entarch/pdf/oea_framework.pdf) from Oracle EA Center

Not able to afford Oracle RAC - Scale out low end / cost DB with Open Source Technology Stack - Tungsten

2009-10-04T23:44:00.000-07:00

Open-source middleware maker released a database scale-out stack called Tungsten, which support open-source databases like MySQL as well as proprietary ones from the likes of Oracle.

The stack's capabilities include a failure protection function that keeps extra database replicas in the event the master fails, and can automatically promote a slave to master status when needed. It also enables users to maintain and replicate database copies at a number of locations to aid disaster recovery.

Scale-out works by spreading data across multiple, independent database servers connected through a network. The model offers an incremental approach to solving the following database problems:

* Preventing data loss though up-to-date replica databases and coordinated backups
* Increasing overall application availability by providing rapid database failover
* Raising performance and through-put by dispatching read traffic to replicas
* Integrating data between heterogenous systems, for example to support scaling of commercial databases using low-cost open source databases.

Scale-out implementation involves a number of complexities, including creating and maintaining replicas, managing resources spread across many hosts, and working around important problems such as maintaining data consistency when databases are spread over multiple hosts.

Tungsten offers a set of components that address these problems in a systematic way.

Tungsten Replicator
Implements database-neutral, master/slave replication.

Tungsten Manager
Group-communication based framework for managing cluster resources.

Tungsten Connector
A proxy for MySQL and PostgreSQL client programs. The Connector serves as a "front door" to clustering logic implemented in JDBC drivers.

Tungsten SQL-Router
A JDBC wrapper that adds high-performance and transparent failover, load-balancing, and partitioning to native JDBC drivers.

Tungsten Monitor
A pluggable monitoring service that checks and broadcasts state of individual resources in the cluster.

Tungsten claimed it isn't trying to supplant high-end Oracle Real Application Clusters (RAC), but instead hopes to provide a potential incremental scale-out offering for customers using low-end databases and hardware that either can't afford or don't want to buy into the whole Oracle RAC architecture.

However, it did provide an alternative low end / cost open source RAC solution for potential Oracle RAC customer and attract some eyeballs over.

Some reflection is What if oracle or its well known competitor come out & buy over similar technology to bridge & guide the smooth transition from low end to high end. Who will be the winner in this arena?

Further readings:

What Is Tungsten

The Scale-Out Blog

Addressing Challenge, Design, Build & Provisioning Saas with Open Source Software Stack

2009-09-08T02:51:00.000-07:00

Challenges & How to Address

When design & build SaaS application, there are some common technical challenges we have to address:

- Multi-tenancy
- Enable a unique user experience.
- Data isolation for SaaS application, either isolated DB or a shared DB, e.g. shared tables
- Security, authentication, authorization and access control of application users & application modules

Check out this demo and learn how IBM tackle them and you will have idea of how you can do it with IBM and other similar technologies.

Build SaaS application with Open Source Software Stack (OSSS)

This sample multi-tenant SaaS application run on a single instance of WAS (Community Edition), DB2 Express-C and openLDAP with a different look-and-feel and access control.

Check out this demo and learn how IBM people build it and you will have idea of how you can do it in your own way.

Provisioning SaaS with ANT & OSSS

This demo focuses on the ease of provisioning new tenant SaaS application through the use of Apache ANT scripts. A new virtual host and security realm are provisioned through ANT scripts invoked from a new service provider administrator portlet. A new openLDAP user database is created and new LDAP Data Interchange Format (LDIF) files imported through another ANT script. The portal for the new tenant application is customized through a tenant administrator portlet by modifying style sheets, providing tenant specific images and uploading and deploying these to the running application. Custom fields are added to other portlets which use XML columns defined in DB2® Express-C V9 through simple configuration steps in a tenant administrator portlet.

Resources

IBM SaaS Solutions & Demonstration Series

IBM Software as a Service (SaaS) Community

IBM SaaS Technical Library

SalesForce Online Training

How to Tackle the SaaS Integration Problem

2009-08-24T04:34:00.000-07:00

This article is written by Loraine Lawson from itbusinessedge

"Integration Issues May Hinder SaaS Adoption," from CIOUpdate, reminds us of the integration problems created by buying "best of breed" applications in the 1980s and '90s. No doubt, you remember it well. And yet the same thing is now happening as companies opt for more software-as-a-service applications.

Just like with those proprietary best-of-breed solutions, this is code that you can't customize. How can you integrate SaaS offerings? Here's an abbreviated version of the options outlined in the article:

Use middleware, sold either as an appliance (Cast Iron Systems) or as an application integration platform, (e.g., Tibco).
Use Web services, which brings challenges with security, data models, business processes and workflow, according to the article.
Buy pre-integrated suites from one vendor. Microsoft, SAP and Salesforce.com are among those migrating to this approach, which relies on a "core" to provide integration for products from the vendor and its partners. For example, SAP's core is Netweaver platform. Oracle's is Fusion. Salesforce.com hopes to leverage AppExchange platform into its core. The article doesn't specify Microsoft's core, but my guess would be Sharepoint server. If you're thinking "Surely companies won't fall for that kind of lock-in," remember: This is what most companies opted to do in the '90s instead of integrating in-house applications.
Buy an EAI solution from a third-party.

Of course, SOA doesn't solve the underlying data definitions problem, as the piece points out. This is where Master Data Management comes into play.

The full article fleshes out the problem and the possible solutions, as well as issuing advice about what IT departments can do now to encourage vendor integration, (yeah, right), and what to do if that doesn't work out.

Here are three related resources to help you learn more:

SaaS integration is a problem that will impact all sizes of IT organizations. Small and mid-sized companies treated SaaS applications as silos from the start, according to integration and SOA consultant and columnist David Linthicum. See what he recommends these businesses do now to remedy the SaaS integration problem by reading "SMBs Need to Tackle SaaS Integration," from Intelligent Enterprise.
Informatica offers an integration solution for organizations using Salesforce.com. Its On Demand Data Replicator allows businesses to easily replicate SaaS-related data to their databases. Learn more by reading, "Service Integrates SaaS Data On Demand," on the blog THINK IT Services.
Find out what's driving Master Data Management adoption and learn how the two breeds of MDM solutions may begin to merge functionalities in our Executive Briefing, "What's Next for Master Data Management?"

Case Study & More:

Appirio’s eat-your-own-dogfood SaaS integration

Three Startups To Solve SaaS Integration Problems

SaaS integration without the lock-in

Building a Business on Virtual Infrastructure, Using Google and salesforce.com

Videos

There are list of videos available from YouTube regarding Saas integration, check out some of them at your spare time.

Cloud Computing , SaaS and Prospects

2009-08-24T02:43:00.000-07:00

Cloud computing is a style of computing in which dynamically scalable and often virtualized resources are provided as a service over the Internet. Users need not have knowledge of, expertise in, or control over the technology infrastructure in the "cloud" that supports them.

The concept generally incorporates combinations of the following:
* infrastructure as a service (IaaS)
* platform as a service (PaaS)
* software as a service (SaaS)
* Other recent (ca. 2007–09) technologies that rely on the Internet to satisfy the computing needs of users.

Cloud computing services often provide common business applications online that are accessed from a web browser, while the software and data are stored on the servers.

The term cloud is used as a metaphor for the Internet, based on how the Internet is depicted in computer network diagrams and is an abstraction for the complex infrastructure it conceals.

Cloud computing users can avoid capital expenditure (CapEx) on hardware, software, and services when they pay a provider only for what they use. Consumption is usually billed on a utility (e.g. resources consumed, like electricity) or subscription (e.g. time based, like a newspaper) basis with little or no upfront cost. (from wikipedia)

Late 2008, market research group IDC surveyed IT professionals and concluded that 4% of enterprises already have implemented some form of cloud computing, although it's often in the form of software as a service (SaaS). That number will more than double by 2012, to 9% of enterprises, said Frank Gens, senior VP of IDC, as he opened the Cloud Computing Forum in San Francisco

Impact of cloud computing will far exceed those modest figures, he added. Applications designed to run in the cloud "will represent 25% of the net new growth in IT spending," versus spending for on-premises IT, he predicted. SaaS by itself is projected to nearly double from $9 billion to $17 billion in that time period, Gens said.

Joseph Tobolski, director of cloud computing at Accenture's Technology Labs, pointed out that "Achieving a 5% to 9% adoption rate through 2012 might be understating it", He said Accenture's cloud consulting services are growing, and it's even produced an enterprise service bus to make it easier for its clients to link their data centers to cloud services. IT managers should be trying to make their internal operations function more as an enterprise cloud. (from InformationWeek)

In current an uncertain economy, An AMI-Partners survey finds SMB are focusing more on software as a service (SAAS) and managed services solutions as IT budgets tighten.
The study’s focus was to analyze how the economy is impacting SMBs’ perceptions, usage and purchasing behaviors related to IT. Among key changes identified in the study was the drastic increase of SMBs worldwide now showing strong interest in managed services and SAAS and dramatic increases in midmarket companies’ plans to outsource specific IT needs such as storage, security and telecommunications.

The study, released this summer, found that most SMBs feel the economy is starting to stabilize, however, businesses are still seeking ways to significantly reduce costs and increase revenues. This includes exploring IT products and services that can directly and immediately help ease exaggerated pain points like restricted cash flow and limited access to credit. “Solutions like SAAS and managed services offer flexible payment options and usage-based models that are very attractive to SMBs right now, as they struggle to overcome the credit crunch and very tight IT budgets,” said AMI Vice President of Marketing Chad Thompson. (from eWeek)

A SaaS Summit 2009, organized in Bangalore has also echoed & presented the trends of SaaS adoption in the current uncertain economy.

"SaaS is the new reality, which enterprises are facing," said M S Krishnan, professor of Business Information Technology at the Ross School of Business, University of Michigan.

"Recession has taught a lesson for restructuring of businesses as the market becomes more and more competing and giving way for newer technologies like SaaS. The social demand for these models have increased which are going to transform the business models," he added.

Balka Baruah Aggarwal, Manager, Syndicated Research, Springboard Research said IT department in enterprises have begun to move to the back-end and the focus is turning on the large scale business goals.

"IT has become the secondary focus," she said. "SaaS plays a vital role in this fundamental shift of enterprises looking at their technology investments, as it allows them to keep the focus on their business without worrying about the technology as well as successfully tweaks into your business process."

The event witnessed the delivery of ERP on a SaaS model as the emerging trend. With no initial investment costs, affordable pricing models as well as flexibility to customization, ERP on SaaS was seen as the next innovative technology by the speakers of the Summit.

Highlighting as an example, A vice president from Indian metal products compay, said, "One of our purposes is not to have any IT team in our company, that is why we went for a web-hosted ERP model. We also realized that the model can be quickly implemented and also scale our future requirements without any huge investments." (from CIOL)

Resources

Grid Asia - 2009

Basic Introduction to Cloud Computing Applications & Systems

Developing a Cloud Ecosystem in Singapore - Singapore IDA's Approach

Cloud computing - WIKIPEDIA

Cloud Implementation To Double By 2012 - InformationWeek

Economy Ignites Interest in Managed Services and SAAS - eWeek

SaaS gaining grounds in downturn - CIOL

Getting Startedwith Cloud Computing - Sun

Microsoft Operations Framework

2009-03-24T02:36:00.000-07:00

The goal of MOF is to provide guidance to IT organizations to help them create, operate, and support IT services while ensuring that the investment in IT delivers expected business value at an acceptable level of risk.

MOF’s purpose is to create an environment where business and IT can work together toward operational maturity, using a proactive model that defines processes and standard procedures to gain efficiency and effectiveness. MOF promotes a logical approach to decision-making and communication and to the planning, deployment, and support of IT services.

The MOF IT service lifecycle encompasses all of the activities and processes involved in managing an IT service: its conception, development, operation, maintenance, and—ultimately—its retirement. MOF organizes these activities and processes into Service Management Functions (SMFs), which are grouped together in lifecycle phases. Each SMF is anchored within a lifecycle phase and contains a unique set of goals and outcomes supporting the objectives of that phase. The SMFs can be used as stand-alone sets of processes, but it is when SMFs are used together that they are most effective in ensuring service delivery at the desired quality and risk levels.

MOF 4.0: it is designed to help IT professionals quickly access useful, relevant content. It contains practical guidance—not just theory—and its streamlined approach makes it possible to use either the entire framework or one process from a particular service management function (SMF).

MOF Job Aids: These templates, worksheets, and tools help IT pros apply MOF 4.0 guidance to specific scenarios and challenges.IT Pro Quick Start Kit. Save time and money by learning more about MOF 4.0.

The IT Pro Quick Start Kit: it gives an overview of MOF 4.0 and provides the tools to start using it today. The kit includes links to presentations and data sheets, information on MOF, compliance, and training and certification, ready-to-use graphics, and supporting materials.

To learn more, visit www.microsoft.com/MOF.

Further reading

ITIL vs. MOF:
Find the best IT framework option for the midmarket

Product Lifecycle Management (PLM)

2009-02-15T04:57:00.000-08:00

What is PLM? - (Excerpt from PLM Technology Guide)

Product lifecycle management or PLM is an all-encompassing approach for innovation, new product development and introduction (NPDI) and product information management from ideation to end of life. PLM Systems as an enabling technology for PLM integrate people, data, processes, and business systems and provide a product information backbone for companies and their extended enterprise.

Business Drivers

Innovation and new product development are essential for most companies to sustain future revenue growth. Customers demand more new products in shorter time intervals, often customized to their own needs. They want more attractive designs, better performance, better quality, lower prices, and instant availability. To meet these needs companies have to be able to collaborate closely within their own organization and with partners and suppliers located in various parts of the world. At the same time companies have to manage increasing product and manufacturing complexities due to a quickly growing number of environmental and regulatory rules and requirements.

The Problem

Accelerating innovation and increasing the number of successful new product introductions is a huge challenge for most organizations today because of their traditionally serial, fragmented, manual, and paper based processes. The result is that many companies suffer from NPDI practices that are slow, resource intensive, costly, inflexible, provide little visibility, and are difficult to manage and control.

The Solution – PLM

Through their ability to integrate all product related data and processes and to eliminate boundaries in the value chain, PLM Systems can significantly reduce non-value added activities and enable stakeholders to collaborate in real time using a consistent set of information throughout the entire product lifecycle.

As a result, productivity improvements of over 60% in NPDI-related activities have been achieved through PLM-enabled, enterprise-wide data and process optimization and integration that have allowed companies to:

Drive innovation
Accelerate Revenues
Increase Productivity
Reduce Costs
Improve Quality
Ensure Compliance
Shorten Time-to-Market

In today’s highly competitive, fast-paced and global business environment, well-designed and implemented PLM practices, processes and technologies that support an organization’s strategies for innovation and growth can afford companies a real competitive advantage.

Reference URLs:

PLM Technology Guide

10 Best Practices for Successful PLM Evaluations

PLM from Wikipedia

Oralce Accquire PLM Company

Unified Process vs Agile Process

2009-02-09T00:56:00.000-08:00

What is UP? Definition from Wikipedia - IBM RUP
What is Agile? Definition from Wikipedia - Agile software development

Comparison of UP & Agile

"
Software development processes define a list of activities to undertake, what order the activities should proceed in, what the inputs and outputs of each activity should be, etc. What few software process specifies is when to stop. It’s often too easy to overuse a process, and continue to produce system designs, resulting in a heavyweight, document centric process. Essentially, process is followed for process sake, rather than adding value to the stakeholders of the project.

Although UP is iterative, it follows the general phases of inception, elaboration, construction, and transition in a more or less linear fashion. As the development proceeds less time is spent on requirements and analysis, and more time is spent on construction, testing, and transition.

Agile processes on the other hand, spend time on each activity in each iteration. During each iteration (usually 2-4 weeks) a little of all steps (requirements, analysis, design, development, test, etc.) are undertaken. The goal is to have stable software which could be shipped at the end of each iteration. At the beginning of each iteration, all stakeholders meet to reorganise requirements and their priorities, while at the end of the iteration, working software is demonstrated to the stakeholders. This ensures that customer value is being added at all times, and that progress is being made on what matters, working software.

UP and its relative the Rational Unified Process (RUP) are quire comprehensive, and were designed to be modified to suit the project being developed. This adds another layer of complexity to the development effort, as the process has first to be modified before being implemented. While this allows for the flexibility to use UP on any project, it requires teams to have access to UP experts to ensure that the process is being defined and used properly.

Agile on the other hand tends to be used for small to medium sized projects involving teams up to 10 closely knit developers. Once teams become bigger than this, agile methodologies begin to fail, as they don’t scale to large teams, or teams spread across geographies." - from chris blog

Which one is better and which one to use?

like what chris said in his blog "there is no silver bullet, or one size fits all approach. You need to tailor the process to the project and your individual circumstances. " Agile strike a good balance between process and productivity, but there is some project out there where agile may be an inappropriate process of choice. RUP is great, but you have to have experienced people doing RUP projects as you don't want to overload everyone with documentation, but also have to carefully look at what the customer wants.

One important difference (from dennis) between UP & Agile is that communication in UP is more through documents where as in Agile it's face to face communcation between the customer and the entire team.

SCRUM - What is SCRUM? Definition from Wikipedia - SCRUM

Scrum is an iterative incremental process of software development commonly used with agile software development.

Although Scrum was intended to be for management of software development projects, it can be used in running software maintenance teams, or as a program management approach.

Scrum is a process skeleton that includes a set of practices and predefined roles. The main roles in Scrum are the "ScrumMaster" who maintains the processes and works similarly to a project manager, the "Product Owner" who represents the stakeholders, and the "Team" which includes the developers.

During each "sprint", typically a 2–4 week period (length decided by the team), the team creates an increment of usable software. The set of features that go into a sprint come from the product "backlog", which is a prioritized set of high level requirements of work to be done.

Scrum enables the creation of self-organizing teams by encouraging co-location of all team members, and verbal communication across all team members and disciplines that are involved in the project.

A key principle of Scrum is its recognition that during a project the customers can change their minds about what they want and need (often called requirements churn), and that unpredicted challenges cannot be easily addressed in a traditional predictive or planned manner. As such, Scrum adopts an empirical approach—accepting that the problem cannot be fully understood or defined, focusing instead on maximizing the team's ability to deliver quickly and respond to emerging requirements.

There are several implementations of systems for managing the Scrum process which range from yellow stickers and whiteboards to software packages. One of Scrum's biggest advantages is that it is very easy to learn and requires little effort to start using.

Scrum Meetings

Daily Scrum
Each day during the sprint, a project status meeting occurs. This is called a "scrum", or "the daily standup". The scrum has specific guidelines:

* The meeting starts precisely on time. Often there are team-decided punishments for tardiness (e.g. money, push-ups, hanging a rubber chicken around your neck)
* All are welcome, but only "pigs" may speak
* The meeting is timeboxed at 15 minutes regardless of the team's size
* All attendees should stand (it helps to keep meeting short)
* The meeting should happen at the same location and same time every day

During the meeting, each team member answers three questions:

* What have you done since yesterday?
* What are you planning to do by today?
* Do you have any problems preventing you from accomplishing your goal? (It is the role of the ScrumMaster to remember these impediments.)

Sprint Planning Meeting
At the beginning of the sprint cycle (every 15–30 days), a "Sprint Planning Meeting" is held.

* Select what work is to be done
* Prepare the Sprint Backlog that details the time it will take to do that work
* 8 hour limit

Sprint Review Meeting
At the end of a sprint cycle, two meetings are held: the "Sprint Review Meeting" and the "Sprint Retrospective"

* Review the work that was completed and not completed
* Present the completed work to the stakeholders (a.k.a. "the demo")
* Incomplete work cannot be demonstrated
* 4 hour time limit

Sprint Retrospective

* All team members reflect on the past sprint.
* Make continuous process improvement.
* Two main questions are asked in the sprint retrospective: What went well during the sprint? What could be improved in the next sprint?
* 3 hour time limit

Please refer to wikipedia for more about role, artifaces in scrum

Scrum is kind of adaptive project management and here are some general practices of Scrum:

* Customers become a part of the development team (i.e. the customer must be genuinely interested in the output.)
* Scrum has frequent intermediate deliveries with working functionality, like all other forms of agile software processes. This enables the customer to get working software earlier and enables the project to change its requirements according to changing needs.
* Frequent risk and mitigation plans are developed by the development team itself—risk mitigation, monitoring and management (risk analysis) occurs at every stage and with commitment.
* Transparency in planning and module development—let everyone know who is accountable for what and by when.
* Frequent stakeholder meetings to monitor progress—balanced dashboard updates (delivery, customer, employee, process, stakeholders)
* There should be an advance warning mechanism, i.e. visibility to potential slippage or deviation ahead of time.
* No problems are swept under the carpet. No one is penalized for recognizing or describing any unforeseen problem.
* Workplaces and working hours must be energized—"Working more hours" does not necessarily mean "producing more output."

Reference URLs:

Agile Modeling and the Rational Unified Process (RUP)

Agile vs. RUP

Unified Process vs Agile Processes

Google Tech Talks - Srcum

EclipseLink

2009-02-06T03:02:00.000-08:00

EclipseLink is an open source object-relational mapping package for Java developers. It provides a powerful and flexible framework for storing Java objects in a relational database or for converting Java objects to XML documents.

EclipseLink's original contribution came from the Oracle TopLink product in 2006.

EclipseLink supports several standard persistence related APIs including:

EclipseLink provides support for:

Object relational mapping (ORM)
Object XML mapping (OXM)
Object persistence to Enterprise Information Systems (EIS)
Database web services

Sun Microsystems has selected the EclipseLink project to be the reference implementation for JPA 2.0.^[1]

Reference List:

Webnar: http://live.eclipse.org/node/490
Wikipedia: http://en.wikipedia.org/wiki/EclipseLink
Introduction of SDO
Introducing EclipseLink
EclipseLink Team Blog

EclipseLink/UserGuide

Java Optical Character Recognition

2009-01-05T03:14:00.002-08:00

Optical character recognition, usually abbreviated to OCR, is the mechanical or electronic translation of images of handwritten, typewritten or printed text (usually captured by a scanner) into machine-editable text.

OCR is a field of research in pattern recognition, artificial intelligence and machine vision. Though academic research in the field continues, the focus on OCR has shifted to implementation of proven techniques. Optical character recognition (using optical techniques such as mirrors and lenses) and digital character recognition (using scanners and computer algorithms) were originally considered separate fields. Because very few applications survive that use true optical techniques, the OCR term has now been broadened to include digital image processing as well.

Early systems required training (the provision of known samples of each character) to read a specific font. "Intelligent" systems with a high degree of recognition accuracy for most fonts are now common. Some systems are even capable of reproducing formatted output that closely approximates the original scanned page including images, columns and other non-textual components. (excerpt from wikipedia)

My understanding for Java OCR implmentation so far is that there is no pure java implementation and most of implementation available in the market is a java porting / wrapper from existing implementation from other language (e.g. c / c++).

You can find list of available softwares from ocr page from wikipedia

Reference List:

Optical Character Recognition

Futurama: Using Java Technology to Build Robots That Can See, Hear Speak, and Move

Optical Character Recognition(OCR)

Java OCR

Java libraries to read and write PDF files

http://www.gnome.sk/

http://asprise.com/home/

http://www.sane-project.org/

JDesktop Integration Components (JDIC)

2009-01-05T03:14:00.001-08:00

JDIC provides Java applications with access to functionalities and facilities provided by the native desktop. It consists of a collection of Java packages and tools. JDIC supports a variety of features such as embedding the native browser, launching the desktop applications, creating tray icons on the desktop, registering file type associations, creating JNLP installer packages, etc (the project supports features such as embedding the native HTML browser, programmatically opening the native mail tool, using registered file-type viewers, and packaging JNLP applications as RPM, SVR4, and MSI installer packages. As a bonus, an SDK for developing platform-independent screensavers is included).

Many new features are contributed as incubator projects from the community.

Reference urls:

JDesktop Integration Components (JDIC) project
Introducing JDIC
Multiple JDIC browsers integrated into Processing sketch

Working with XML

2008-10-15T04:16:00.000-07:00

When work with XML, most of time we start with a question of which technology or library should be used to speed up the development and also server the biz spec.

One guy summarized it in an article and here is the excerpt:

Which tool should I pick?

If you're new to XML you might not quite understand the need for having several XML tools. You might think that an API that allows you to read and write an XML document would be sufficient. The need for several tools arises when you realize that XML can be used in a lot of very different situations. Some examples are:

configuration files--or parameter files--are often written in XML. They typically contain technical data for systems like web servers.
XML files can be used when sending updates of business data to an application. Think of a list of new customers, orders or exchange rates.
XML data sometimes have to be persisted on files or in databases, but more often XML is only a format held in memory.
XML can be used as a description language for other types of XML documents. An example is XML schemas that are used to validate other XML documents.

Let's look at some of the different situations where you need to read an XML document, for example from a file.

When reading a configuration file, you need to get some or all of the parameters extracted from the file and then stored somewhere in your program. Digester is a great tool for this kind of operation. An introduction to Digester is "Digesting XML documents". Digester is also a natural choice if you only want to read some of the data in an XML file.

If you want to read all the data in an XML file and then continue to do some processing with these data, then you need to convert the XML into some kind of Java object structure. The simplest way to do this is to convert into JavaBeans, since beans are simple to work with. My own favorite tools for converting between XML files and JavaBeans are Castor and XMLBeans. Two articles that give you more information are " Converting XML documents to Java objects with Castor XML" and "Converting XML to JavaBeans with XMLBeans". XMLBeans is currently my number one pick since it seems capable of handling even the most complex XML structures and automatically packs the generated JavaBeans into a jar-file.

Castor and XMLBeans can be used when your XML files have a known, fixed structure that doesn't change over time. Castor and XMLBeans typically take the XML schemas that define the XML structures and generate beans for you. This is something you'll have to do manually, so you need an XML format that doesn't change too often.

If the format of your XML data is not fixed, you need to look for another tool. JDOM is such a tool, since it'll read any XML file and produce a tree-like structure in memory. This structure is very easy to work with for a Java programmer because of its simple API and its use of well-known Java objects like those implementing the List interface.

"

For full article, read from here .

Additional reading list:

Working with JDOM, XPath and XSLT

Java and JDOM: the perfect couple

Parsing an XML Document with XPath

Java + Excel XLS

2008-10-14T02:15:00.000-07:00

Currently, I am doing a research on various solutions available for java & excel xls integration to be used for our next generation sales quotation tools. I found this article already did some work and compile a list of solution available below:

Library / package	License	Description
Actuate Spreadsheet Engine	Commercial, 30-day trial version available	Write Excel (XLS) files.
ExcelAccessor	?	Read and write Excel (XLS) files.
ExcelReader	?	JavaWorld article on how to read Microsoft Excel (XLS) files, including code. Requires an installed Microsoft ODBC driver for Excel files, and Sun's ODBC-JDBC driver.
ExtenXLS	Commercial, 30-day trial version available	Read and write Microsoft Excel (XLS) files.
JACOB project	LGPL	Java COM bridge with examples to interface Excel.
Java Excel API	LGPL	Read Excel (XLS) 97, 98 and 2000 files.
Java to Excel conversion	?	Write SYLK files, which Excel can import and export.
JExcel	Commercial	Swing component to integrate Excel via JNI.
jXLS	LGPL	Create Excel files using XLS templates.
POI	Apache Software License 1.1	Read and write Microsoft OLE 2 compound document format files. This includes MS Office files (DOC, XLS, PPT) written with Office versions that were released after 1997.
Snowbound	Commercial	Read Excel files.
SoftArtisans OfficeWriter	Commercial	Write Word and Excel files.
Vista JDBC	Commercial, 15-day trial version available	JDBC driver to access MS Excel (XLS) files.
xlSQL	GPL	JDBC driver to access MS Excel (XLS) and CSV files.

In addition to this table, following links are viable choice for java & excel integration in the application.

Handle Excel files

Content Tagged with excel + Java

ExtenXLS Java Spreadsheet SDK

XLSBeans - mapping Excel sheets to POJOs

Java Excel API

JACOB - JAVA-COM Bridge to call COM components

The Java-Excel solution

Read and Write Microsoft Excel Documents with Jakarta's POI - (The Excel Document Model)

Excel Reports with Apache Cocoon and POI

Jxcell

j-integra - Accessing Excel from Java

ActiveXLS™ Professional for Java

Apache POI - Java API To Access Microsoft Format Files

jXLS - generating Excel files using XLS templates

Cloudspace

2008-06-12T20:25:00.000-07:00

As one of our project is looking for a open source, lightweight and stable database to be embedded into application, cloudspace has come into my eyes as its superior features and ready support of synchronization with various enterprise back ends, such as DB2 UDB™ or Informix® Dynamic Server.

Cloudscape is the original zero-admin, embeddable, 100% Java relational database that entered the marketplace in 1996. In August 2004 IBM contributed Derby, a copy of its Cloudscape 10.0 relational database product, to the Apache Software Foundation (ASF) to help accelerate innovation around data-driven Java applications. IBM continues its Cloudscape commercial offering at no-charge, which adds features to the core Derby engine. The most current release is Cloudscape 10.1, which includes Apache Derby 10.1.

Cloudscape- Key Features

* 100% Pure Java
o Completely portable to any hardware, any OS
o Only requires JVM (J2SE 1.3 or higher)
* Embeddable
o Can be installed as a JAR file (as part of a Java application)
o Can be started within an application
o Installation, startup and shutdown are invisible to user
* Fully-functional RDBMS

* Adheres to standards
o JDBC
o SQLJ
o ANSI SQL-92 * "Zero" Administration
o No need for DBA at customer site
o Users may even be unaware that a database engine is present
* Lightweight
o Approx. 2MB “footprint”
o Can run on many “palm”-size devices (new feature!)
* Scales to Moderate Size
o No defined limits on table sizes
o Table size limit of 2GB in some environments (OS limit)
o Existing customer databases of >10 GB working well
* Supports Disk Encryption
o Tested with IBM JCE

* Easy for Application Developers to Use
o To set up, just copy Cloudscape JAR files and set Java CLASSPATH
o Database engine starts automatically when application attempts to connect to it
* Cloudscape databases are completely portable
o To move a Cloudscape database to another machine, just copy it! (This works even if the machines have different OS or hardware!)
* Distributed transaction support (new!)
o Participates in XA transactions with other database systems

Cloudscape is a Fully-Functional RDBMS

* Indexes
* Views
* Triggers and Stored Procedures
* Constraints (primary-key, foreign-key, unique, check)
* Concurrency Support
o Transactions (with crash recovery)
o Savepoints
o Row-level locking, with escalation
o JDBC isolation levels (including dirty-read)
* Scrollable Cursors (result sets)
* Cascading Delete
* Internationalization/Localization Support

Reference List:

Synchronize IBM Cloudscape V10.0 and IBM DB2 Everyplace Sync Server V8.2

DB2 Everyplace Java Sync Client for IBM Cloudscape Version 10

Cloudscape Past and Present

Cloudscape Version 10: A technical overview

TPC - Transaction Processing Performance Council

2008-06-12T19:51:00.000-07:00

An organization devoted to benchmarking transaction processing systems. In order to derive the number of transactions that can be processed in a given time frame, TPC benchmarks measure the total performance of the system, which includes the computer, operating system, database management system and any other related components involved in the transaction processing operation.

Transactions
The TPC-C benchmark yields transactions per minutes expressed in tpmC ratings. Earlier TPC-A and TPC-B benchmarks produced tpsA and tpsB ratings, which measured transactions per second.

TPC-W is a Web benchmark, performed in a controlled Internet e-commerce scenario and reported as the number of Web interactions processed per second. Shopping, browsing and ordering are the three simulated profiles measured.

Database Queries
TPC-H carries an ad-hoc label and is used in environments where users do not know which queries will be executed against a database system. TPC-R analysis assumes pre-knowledge of a standard set of queries, and execution is very fast as a result. The earlier TPC-D benchmark based its results on the size of the database being queried and yielded a single-user Qppd Power metric and multiple-user QthD Throughput metric.

TPC benchmarks are widely used today in evaluating the performance of computer systems; the results are published on the TPC web site.

Reference

TPC Intro
Wikipedia
TPC Portal

open source audit trail and logging component

2008-06-11T18:11:00.000-07:00

The Inspektr project allows for non-intrusive auditing and logging by using annotations and aspects.

It also includes a generic Grails-based web application to view statistics and audit logs stored in various data sources.

Find out more from here

Groovy, Java's New Scripting Language

2008-06-11T04:25:00.000-07:00

Groovy is a new scripting language for the Java environment. Groovy scripts can be interpreted (for faster development: no compile cycle!) or compiled to Java bytecode (.class format) for faster execution. Groovy accepts most Java constructs but also quite a few of its own.

* is an agile and dynamic language for the Java Virtual Machine
* builds upon the strengths of Java but has additional power features inspired by languages like Python, Ruby and Smalltalk
* makes modern programming features available to Java developers with almost-zero learning curve
* supports Domain-Specific Languages and other compact syntax so your code becomes easy to read and maintain
* makes writing shell and build scripts easy with its powerful processing primitives, OO abilities and an Ant DSL
* increases developer productivity by reducing scaffolding code when developing web, GUI, database or console applications
* simplifies testing by supporting unit testing and mocking out-of-the-box
* seamlessly integrates with all existing Java objects and libraries
* compiles straight to Java bytecode so you can use it anywhere you can use Java

See these links for complete information on Groovy:

http://www.darwinsys.com/groovy/

Groovy Is Not Java

Differences from Java

http://groovy.codehaus.org/

Java on Grails ?

2008-06-11T03:54:00.000-07:00

Java programmers needn't abandon their favorite language and existing development infrastructure to adopt a modern Web development framework. In the first installment of his new monthly series Get started with Grails, get introduced to Grails and demonstrates how to build your first Grails application.

Grails is a modern Web development framework that mixes familiar Java technologies like Spring and Hibernate with contemporary practices like convention over configuration. Written in Groovy, Grails gives you seamless integration with your legacy Java code while adding the flexibility and dynamism of a scripting language. After you learn Grails, you'll never look at Web development the same way again.

Find original post here

Run your web applications on Google's infrastructure!

2008-06-11T03:49:00.000-07:00

Google App Engine lets you run your web applications on Google's infrastructure. App Engine applications are easy to build, easy to maintain, and easy to scale as your traffic and data storage needs grow. With App Engine, there are no servers to maintain: You just upload your application, and it's ready to serve your users.

You can serve your app using a free domain name on the appspot.com domain, or use Google Apps to serve it from your own domain. You can share your application with the world, or limit access to members of your organization.

App Engine costs nothing to get started. Sign up for a free account, and you can develop and publish your application for the world to see, at no charge and with no obligation. A free account can use up to 500MB of persistent storage and enough CPU and bandwidth for about 5 million page views a month.

During the preview release of Google App Engine, only free accounts are available. In the near future, you will be able to purchase additional computing resources.

Find more from here

Make Windows Installer for your application

2008-04-10T19:38:00.000-07:00

Recently, my project is using a window installer script called NSIS to make windows executable exe file for product distribution which brings me to a post that comparing the similar products out there.

"Many of you know that Windows Installer is the preferred method of distributing software, according to Microsoft. However, Windows Installer has the serious flaw of requiring the OS to support the Windows Installer Runtime. This can be a problem on older operating systems, such as Windows 95 Gold and Windows NT SP0-SP3. ... If your program is designed to work on all Win32 systems, then you cannot use Windows Installer. You could use something such as Inno Setup or Nullsoft Scriptable Install System. Both are very good, but I believe that NSIS is much more versatile and expandable, due to its widespread support and many various plugins to make it more professional. The only thing NSIS is missing is Visual Studio plugins. At least for VS6 it would be good, since the InstallShield for VC++6 is very bad. Also, many installers that I built with that version of InstallShield do not work on XP due to the old structure of the installer. If you do not want to shell out $300+ for a good installer with universal support and still have a professional look, I recommend three things in combination: NSIS, HM NIS Edit, and ExperienceUI SDK. All three together allow you to build installers that look as good as the latest InstallShield installers, but with smaller overhead and much more flexibility. I see more good coming from open-source each day. NSIS is currently at 2.14 release, HM NIS Edit is at 2.0.3, and ExperienceUI SDK is at 1.1 release. The major advances in all three make things easier for the lower budgeted programmers to make nice installers for their projects. I think that Inno Setup should get its fair end of praise. Inno Setup is designed to be very similar in style to InstallShield, and does a good job of it. Inno Setup's ability to go silent install is very useful for automated runtime installs, making it as good as an automated Windows Installer installation. However, NSIS has advanced SILENT ONLY capabilities, forcing network install styles. Inno Setup already has 64-bit installations, while most others do not yet have this capability. I think NSIS may be able to be recompiled to build 64-bit installs, but I am not quite sure on that. InstallShield has one advantage that would make someone who needs this feature to shell out for it: The ability to make cross-platform installers. InstallShield can make RPMs for Linux apps and binary installers for Windows, MSI files for Windows Installer, and more. That would be the only reason I would shell out for it, if I had a program developed cross platform and need easy installer development. However, recently I came across another installer: InstallerBuilder Enterprise... This version does the one thing that would have tied me to InstallShield, ....
"

References

Find the original post (1 | 2)

External Links:

NSIS Users Manual

Windows Installer Authoring Tools

MSI / EXE UnAttended Installation

JTS Topology Suite & GeoTools

2008-01-17T23:10:00.000-08:00

Both are java open source tools for spatial data management.

The Open Source Java GIS Toolkit

The JTS Topology Suite is an API of 2D spatial predicates and functions. It has the following design goals:

* JTS conforms to the Simple Features Specification for SQL published by the Open GIS Consortium
* JTS provides a complete, consistent, robust implementation of fundamental 2D spatial algorithms
* JTS is fast enough for production use
* JTS is written in 100% pure JavaTM
* JTS is open source (under the LGPL license)

Find more information, you can find it from here.

Geo Tools is an open source (LGPL) Java code library which provides standards compliant methods for the manipulation of geospatial data, for example to implement Geographic Information Systems (GIS) . The Geo Tools library implements Open Geospatial Consortium (OGC) specifications as they are developed, in close collaboration with the GeoAPI and GeoWidgets projects. The capabilities of Geotools are presented in the feature list.

For more information, you can find it from here ( 1|2 ).

Understanding insurance

2008-01-17T20:21:00.000-08:00

Having this post is because i am working some projects for insurance and need to have the domain knowledge of insurance.

Term life insurance

Term life insurance is the original form of life insurance and is considered to be pure insurance protection because it builds no cash value. This is in contrast to permanent life insurance such as whole life, universal life, and variable universal life.

Permanent life insurance

Permanent life insurance is a form of life insurance such as whole life or endowment, where the policy is for the life of the insured, the payout is assured at the end of the policy (assuming the policy is kept current) and the policy accrues cash value.

Here are the main characteristics of permanent life insurance:

* Permanent insurance protection.
* More expensive to own.
* Builds cash value.
* Loans are permitted against the policy.
* Favorable tax treatment of policy earnings.
* Level premiums.

There are three basic types of permanent insurance: whole life, variable life and universal life. The two most common are whole life and universal life. Whole life insurance provides lifetime protection, for which you pay a predetermined premium. Cash values usually have a minimum guaranteed rate of interest and the death benefit is a fixed amount. Whole life insurance is the most expensive life-insurance product available.

Universal life insurance separates the investment and the death benefit portions. The investment choices available usually include some type of equity investments, which may make your cash value accumulate quicker. As the you can usually change your premiums and death benefits to suit your current budget.

What i feel that variable and universal life insurance are all variation of whole life insurance policy and differences are:

Variable life insurance policy is something like death benefit + flexible investment options and much more expensive and risky as there are investment risk.

Universal life insurance policy is something like death benefit + saving component + loan facility as insurer will pay fixed amount regularly against to the interest rate or certain managed fund. It is much cheaper than whole and variable life insurance policy.

You can find more from here

Participating Policy

An insurance contract that pays dividends to the policy holder. Dividends are generated from the profits of the insurance company that sold the policy and are typically paid out on an annual basis over the life of the policy. Most policies also include a final or terminal payment that is paid out to the holder when the contract matures. Some participating policies may include a guaranteed dividend amount, which is determined at the onset of the policy.

Also referred to as a "with-profits policy".

Investopedia Says:
Participating policies are typically life insurance contracts such as a whole life participating policy. The dividend received by the policy holder can be used in several different ways. First, the policy holder can apply the dividend proceeds to the insurance policy's premium payment. Second, the dividend can be kept with the insurance as a deposit in order to generate interest much like a savings account at a bank. Finally, the policy holder can simply receive the dividend payment in cash, much like a dividend payment on a stock.

Nonparticipating Life Insurance Policy

Life insurance policy that does not pay dividends. Policyholders thus do not participate in the interest, dividends, and capital gains earned by the insurer on premiums paid. In contrast, Participating Insurance Policies pay dividends to policyholders from earnings on investments.

Frequently used Insurance Term

Term Life Insurance :: definition
Noted, there is no cash value with this policy.

Permanent Life Insurance :: definition
There are cash values with this set of policies.

Substandard Premium Rate :: definition

This is an extra premium (a substandard premium rate) charged a policy owner as a result of the insured being classified as having a higher-than-average risk of death. Reasons may include the person's physical condition, health, occupation and/or life-style. The premium rate charged is higher than a standard premium rate.

Cash Surrender Value :: definition

Indemnity :: definition

Long Term Care Insurance :: definition

Projected Dividend

Estimated future dividends to be paid by a Participating Insurance Policy. These dividend estimations cannot be part of the policy since they are not guaranteed. They are normally shown in a separate computer printout and are only as accurate as the basic interest rate assumptions made.

Reference and recommended links

Let Life Insurance Riders Drive Your Coverage?

Five Insurance Policies Everyone Should Have

Fifteen Insurance Policies You Don't Need

Understand Your Insurance Contract

Exploring Advanced Insurance Contract Fundamentals

Insurance Glossary (1 | 2)

Why Wiki , Blogging & Social Computing

2007-11-20T00:26:00.000-08:00

Blogging and wiki is something about the knowledge management, one step further than the information management. There are many ways of knowledge management, blogging and wiki are just one of them.

why we bother to blog or wiki it rather than make a phone call, send a email or book a meeting room? Like it or not, social computing is part of how organization works today and an even larger part of how it will work tomorrow.

Social computing is what happens when people selectively consume, produce, and share online content in new ways. Technologies like blogs, peer-to-peer networking, RSS, wikis, and other collaboration tools continue to grow, evolve, and feed the demand for new methods of interaction. The challenges that you face today as a result of access to online information, software, and communities will intensify, with new challenges emerging over time. To maintain influence in this new world, you must embrace social computing, not fight it. You must become a facilitator for this kind of collaboration and a full member of the social computing community.

Develop your interest in using these forces to do your job more effectively, rather than letting them just make your job more difficult. Instead of seeing this as a new onslaught of back-door paths for unwanted change to reach your organization, adopt a leadership role in using these tools to create a more collaborative and productive organization.

Don't fight this trend, embrace it. You should become an advocate and facilitator for social computing. Make sure people can get what they need within the organization so that they won't have to go elsewhere. Make yourself part of the social computing community, and use that insider advantage to help the organization succeed.

As we take advantage of social computing, don't neglect traditional means of communicating and promoting collaboration. Face-to-face idea sharing remains an important tool in building and nurturing critical relationships, and we should continue to use it for communication.

KM Tool - Wiki

2007-11-20T00:19:00.000-08:00

Wikis are tools that allow for collaborative editing and publishing and hence the creation of user generated content. Wikis are another example of social software - software that encourages people to work together.

Wikis are a useful tool when you want people to collaborate and work together on creating a common document, e.g. a project document, a manual, and so on or to work on a common strategy or event, e.g. a conference programme.

In corporate and general use they can be used to create collaborative documents, manuals e.g. WikiHow (a collaborative Manual)

The most obvious and famous example of a wiki is of course Wikipedia, which is an encyclopaedia that has been created by thousands of individuals. Every page in Wikipedia can be edited, which means it is constantly evolving and growing and can be kept up to date.

In the commercial tools market there are a number of enterprise wiki tools available, for example, e.g. Confluence

But there are also a number of free tools, for example:

* MediaWiki is the software used for Wikipedia and the other Wikimedia Foundation websites. Compared to other wikis, it has an excellent range of features and support for high-traffic websites using multiple servers (Wikipedia peaks in the 2500+ requests per second range as of June 2005).
* pbWiki is another hosted service and allows you to set up a wiki very quickly, as are Wikispaces and Wetpaint
* If you are looking for a personal, notepad-type wiki to download onto your own computer, then TiddlyWiki is a useful tool

Reference list

Mediawiki Installation & File Upload HowTO
http://www.washington.edu/computing/web/publishing/mediawiki.html

MediaWiki Comparison Matrix
http://www.wikimatrix.org/show/MediaWiki

How to install PHP and MySQL
http://www.ozzu.com/ftopic56303.html

Wikis - and how to create them quickly and easily
http://c4lpt.co.uk/handbook/wikis.html

SOA, Web Services & BPEL - Tech Forum in Singapore

2007-11-05T22:21:00.000-08:00

Recently, i gave a talk and BPEL demo to singapore partners for Oracle BPEL & SOA suites. Introducing them idea of SOA, BPEL and Web Services and how to make use of oracle technology in their applications.

SOA stands for service oriented architecture, it is a architecture model.

Talk about architecture, we need to understand basic building block of each application and system.

Like we build a real house, we use mortar and brick. Every application / system was built with business logic and process. Business process weaves all business logics into one application to achieve business goals. Process and logic weave seamlessly and hard coded into a monolithic application. Like we cook soup, we put all ingredients into water and hard for us to separate each out in the end, by which it reduce reusability of standard process and logics.

In traditional application, when people wanna change the process or integrated with other application / system, especially it is a heterogeneous platform or technology integration, that will be very hard and time-consuming.

For SOA applications, applications also build with process and so-called business logic (have been wrapped into self-contained services). Process and services are loosely coupled, people can easily add or remove services from process, not like traditional applications (process & logic fuse into one). Other than that, people can easily design & program the business process with a advanced IDE. Overall experience for process design and implementation will be a drag-and-drop exercise.

SOA process should be standard, easily extensible and composable. When build an SOA application, we start from building a portfolio of services , follow with well-defined business process to orchestrate services to achieve business goal. This is where BPEL comes in for business process implementation and web service for business service implementation.

Many people equate web services with Service-Oriented Architectures (SOA), but web services are just one element of SOA - they're about service invocation. SOA's scope is broader than just invocation. It's a complete software component architectural model, a way to organize, manage, deploy, discover, and expose software components, and to consume these over the Internet. From the software life-cycle perspective, in addition to the typical software development phases of design, development, and deployment, SOA defines web services-specific phases that include service publishing, discovery, binding and consumption, and management.

Note, SOA != web service+bpel

ESB vs BPEL

2007-11-05T02:22:00.000-08:00

Both ESB and BPEL can be used for SOA implementation.

ESB (Enterprise Service Bus) implement messaging to enable services to be integrated in a message-based paradigm: both synchronous and asynchronous styles. ESB moves data via different adaptors (WebServices, FTP,File,JDBC etc) and protocols (HTTP,JMS) and enriches and transform data using XSL & domain value mapping lookups.

Oracle BPEL provides a comprehensive, standards-based and easy to use solution for creating, deploying, and managing cross-application business processes with both automated and human workflow steps.

ESB is good for routing messages to multiple destinations. It is also good for doing transformations that have little to no business rules. The footprint is much smaller and incurs minimal overhead therefore the performance is much better.

BPEL is used for bringing together multiple services. There is much more functionality and allows implementation of complex business logic.

• BPEL data transformations enrich and perform complex changes whereas ESB perform only simple transformations.
• BPEL can use Business rules, Human Workflow and Notifications, exception handling can be done in BPEL.

Use ESB when you want a really low cost solution, Only need connectivity, simple transformations and routing.

BPEL is primarily used for servies orchestration, data enrichment and human interaction.

Use BPEL+ESB when, You need to code complex business logic, integration with Workflow, complex transforms potentially involving the database or having long running processes which may or may not be stateful.

With an appropriately layered SOA architecture, one would look at ESB as a repository of services that BPEL orchestrates. At deployment time, this layered approach provides you the flexibility to dedicate servers to individual tiers (BPEL and ESB) and thereby maximize the resource usage for your particular use case.

Io exception: The Network Adapter could not establish the connection

2007-11-02T00:33:00.000-07:00

Recently, i did an RAC validation with our oracle partner and encountered this weird exception called "Io exception: The Network Adapter could not establish the connection".

Two client machine were running same java code (using jdbc thin driver & oracle datasource) in the same LAN as RAC servers, but they came out with differen behaviour. One could connect to RAC database, the other one always got io exception mentioned above.

The only difference between two machines was one of clients' host file having rac ip & hostname mapping entry, which is able to connect to oracle rac db.

Something weird is that all db clients and servers were actually sitting in the same LAN. I could ping and telnet db rac with rac ip address (virtual and real ) from client machines, but when i use ip address in the java class to lookup the db connection, i could not get connected to db if rac ip & hostname mapping entries were not available in the client host file.

Here is the sample Codes for RAC FCF connection:

OracleDataSource ods = new OracleDataSource();
OracleConnectionCacheManager occm = null;
ods.setUser("user-name");
ods.setPassword("password");
String cacheName = "cache";
String dbURL="jdbc:oracle:thin:@"+
"(DESCRIPTION=(ADDRESS_LIST=(LOAD_BALANCE=ON)" +
"(ADDRESS=(PROTOCOL=TCP)" +
"(HOST=rac1-vip-address)(PORT=1521))" +
"(ADDRESS=(PROTOCOL=TCP)" +
"(HOST=rac2-vip-address)(PORT=1521)))" +
"(CONNECT_DATA=(SERVICE_NAME=orcl.abc.com)))";
ods.setURL(dbURL); System.out.println("Url=" + dbURL ); ods.setConnectionCachingEnabled(true);
Properties prop = new Properties();
prop.setProperty("MinLimit", "5");
prop.setProperty("MaxLimit", "40");
prop.setProperty("InitialLimit", "10");
prop.put (oracle.net.ns.SQLnetDef.TCP_CONNTIMEOUT_STR,"" + (5000)); // 5000ms ods.setConnectionCacheProperties(prop);
ods.setFastConnectionFailoverEnabled(true);
ods.setONSConfiguration("nodes=rac1-ip-address:6200,rac2-ip-address:6200"); ods.setConnectionCacheName(cacheName);
//occm = OracleConnectionCacheManager.getConnectionCacheManagerInstance();
Connection conn = ods.getConnection();

It looks somewhere JDBC driver looking up db connection with rac hostname instead of ip address and I have to put the rac ip & hostname mapping in the client host file for jdbc driver to resolve them.

Web service security & clustering

2007-10-31T23:08:00.000-07:00

The Oracle Web Services Manager (OWSM) secures Web services by using policy enforcement points. It has several types of policy enforcement points: Gateways, Web service client agents (client-Agents), and Web service agent (server-Agents). These policies are created, managed, and located in the OWSM Policy Manager, which is a centralized policy store.

An example policy could be to require authentication and authorization to all services. This policy is centrally stored in the Policy Manager, and is consumed by all the agents and gateways deployed in the SOA.

Gateway Use Case

An Internet facing Web service can be protected by inserting a gateway in front of it. A gateway is similar to a web based proxy, it is a choke point where all Web service traffic is forced through. Gateways are SOAP/XML intermediaries that enforce Web services policies while intermediating Web services traffic between clients and services.

This would be done using a WS-Security Username Token for authentication. Based on the credentials, the gateway should be able to authenticate and authorize access to the service.

Alternatively, WS-Security SAML Token can be applied when business partners expose business processes to each other within a secure context.

Agent Use Case

A Web service agent is another useful policy enforcement point. Agents (client-Agents and server- Agents) are SOAP interceptors that enforce Web services policies from within the same Web application.

Agents execute in the same process as the application, while Gateways run on separate processes and possibly on different servers. Gateways can manage services from multiple applications while agents control services belonging to the single application.

Web service is not exposed over the internet, then using agents is a viable option to secure the business process. There can be other reasons for using agents. For example, if one wants end-to-end security where the data is secured from the beginning to the end of the process.

OASIS WSS UsernameToken Profile (User name and password)

OASIS WSS X509 Token Profile 1.1 (certificate based)

OASIS WSS SAML Token Profile 1.1 (Xml accertation token from accertation server for sso)

OASIS WSS SWA Profile 1.1

Reference:

Web Services Security, Part 1

Web Services Security, Part 2

Web Services Security, Part 3

Apache WSS4J 1.1.0 Released

Securing Web Services and the Java WSDP 1.5

Tutorial

Securing Web Services Using the SAML or UserNameToken Profiles

Clustering and Securing Web Applications: A Tutorial

Web Services Security (2003)

http://www.xml.com/pub/a/ws/2003/03/04/security.html
http://www.xml.com/pub/a/ws/2003/04/01/security.html

Oracle TAF & FCF

2007-10-31T02:43:00.000-07:00

Transparent Application Failover (TAF)

Transparent Application Failover (TAF) is a feature of the Java Database Connectivity (JDBC) Oracle Call Interface (OCI) driver. It enables the application to automatically reconnect to a database, if the database instance to which the connection is made fails. In this case, the active transactions roll back.

Transparent Application Failover (TAF) is a client-side feature that allows for clients to reconnect to surviving databases in the event of a failure of a database instance. Notifications are used by the server to trigger TAF callbacks on the client-side.

TAF is configured using either client-side specified TNS connect string or using server-side service attributes. However, if both methods are used to configure TAF, the server-side service attributes will supersede the client-side settings. The server-side service attributes are the preferred way to set up TAF.

TAF can operate in one of two modes, Session Failover and Select Failover. Session Failover will recreate lost connections and sessions. Select Failover will replay queries that were in progress.

Transparent Application Failover
Configuring Transparent Application Failover

Fast Connection Failover (FCF)

Fast Connection Failover offers a driver-independent way for your Java Database Connectivity (JDBC) application to take advantage of the connection failover facilities offered by Oracle Database 10g. The Fast Connection Failover mechanism depends on the implicit connection cache feature. As a result, for Fast Connection Failover to be available, implicit connection caching must be enabled.

FCF vs TAF

TAF is always active and does not have to be set, it does not work with the OCI Connection Pool.
Oracle recommends not to use TAF and Fast Connection Failover in the same application

Web Services Protocol Stack

2007-10-26T00:14:00.000-07:00

You may hear web services for many years and already get some ideas of it, but always get confused and lost in the set of protocol and technology in use and not sure how they are related to each other in the picture.

Here, i would like to give another explanation of them from underlying protocol stack in use for web service so that you can organized, relate them properly in mind.

The Web service protocol stack is a collection of computer networking protocols that are used to define, locate, implement, and make Web services interact with each other. The Web service protocol stack mainly comprises four areas (source:wikipedia):

* Transport Protocol: responsible for transporting messages between network applications and includes protocols such as HTTP(s), SMTP, FTP, as well as the more recent Blocks Extensible Exchange Protocol (BEEP). Now, most common protocol in use is HTTP(S).

* Messaging Protocol: responsible for encoding messages in a common XML format so that they can be understood at either end of a network connection. Currently, this area includes such protocols as XML-RPC, WS-Addressing, REST and SOAP, which is most common one in use today.

* Description Protocol: used for describing the public interface to a specific web service. The WSDL interface format is typically used for this purpose.

* Discovery Protocol: centralizes services into a common registry such that network web services can publish their location and description, and makes it easy to discover what services are available on the network. At present, the UDDI API is normally used for service discovery.

Web service protocol stack also includes a whole range of recently defined protocols: BPEL, SOAP-DSIG.

In general terms, a web service is a software component or service that has been published on the World Wide Web in one of several particular ways, but industry tends to use the term web services more specifically, to describe services that are exposed using SOAP, an XML-based message encoding that is typically transported using HTTP.

XML-RPC

XML-RPC is a remote procedure call protocol which uses XML to encode its calls and HTTP as a transport mechanism. XML-RPC is a very simple protocol, defining only a handful of data types and commands, and the entire description can be printed on two pages of paper. Some people still prefer XML-RPC to SOAP because of its simplicity, minimalism, and ease of use. (source:wikipedia)

SOAP

SOAP is a protocol for exchanging XML-based messages over computer networks, normally using HTTP/HTTPS. SOAP forms the foundation layer of the Web services stack, providing a basic messaging framework so that more abstract layers can build on.

There are several different types of messaging patterns in SOAP, but by far the most common is the Remote Procedure Call (RPC) pattern, in which one network node (the client) sends a request message to another node (the server) and the server immediately sends a response message to the client.

SOAP is the successor of XML-RPC, though it borrows its transport and interaction neutrality and the envelope/header/body from elsewhere, probably from WDDX (source:wikipedia).

A SOAP message is an ordinary XML document containing the following elements:

* A required Envelope element that identifies the XML document as a SOAP message
* An optional Header element that contains header information
* A required Body element that contains call and response information
* An optional Fault element that provides information about errors that occurred while processing the message

All the elements above are declared in the default namespace for the SOAP envelope (source: w3schools)

WSDL

The Web Services Description Language (WSDL) is an XML-based language that provides a model for describing Web services.The current version of the specification is the 2.0. The WSDL defines services as collections of network endpoints, or ports. WSDL specification provides an XML format for documents for this purpose.A port is defined by associating a network address with a reusable binding, and a collection of ports define a service.

WSDL is often used in combination with SOAP and XML Schema to provide web services over the Internet. A client program connecting to a web service can read the WSDL to determine what functions are available on the server. Any special datatypes used are embedded in the WSDL file in the form of XML Schema. The client can then use SOAP to actually call one of the functions listed in the WSDL.WSDL 1.2 was renamed WSDL 2.0 because of its substantial differences from WSDL 1.1. By accepting binding to all the HTTP request methods (not only GET and POST as in version 1.1) WSDL 2.0 specification offers a better support for RESTful web services, much simpler to implement by beginners. (source:wikipedia)

A WSDL 1.1 document uses the following elements in the definition of network services:

* Types– a container for data type definitions using some type system (such as XSD).
* Message– an abstract, typed definition of the data being communicated.
* Operation– an abstract description of an action supported by the service.
* Port Type–an abstract set of operations supported by one or more endpoints.
* Binding– a concrete protocol and data format specification for a (not more than one) particular port type.
* Port– a single endpoint defined as a combination of a binding (not more than one) and a network address.
* Service– a collection of related endpoints (port).

WSDL 2.0 is different from WSDL 1.1 check out the example of a structured WSDL 2.0 document from here.

UDDI

Universal Description, Discovery and Integration (UDDI) is a platform-independent, XML-based registry for businesses worldwide to list themselves on the Internet. UDDI is a cross-industry effort driven by all major platform and software providers like Dell, Fujitsu, HP, Hitachi, IBM, Intel, Microsoft, Oracle, SAP, and Sun, as well as a large community of marketplace operators, and e-business leaders.

JAX-RPC
Check out my previous two posts (1|2) in this blog.

JAXR

Java API for XML Registries (JAXR) defines a standard API for Java platform applications to access and programmatically interact with various kinds of metadata registries. JAXR provides a uniform and standard Java API for accessing different kinds of XML-based metadata registry.

Current implementations of JAXR support ebXML Registry version 2.0, and UDDI version 2.0. More such registries could be defined in the future.

The JAXR API insulates application code from the underlying registry mechanism. When writing a JAXR based client to browse or populate a registry, the code does not have to change if the registry changes, for instance from UDDI to ebXML.

WS-I
The WS-I Basic Profile (official abbreviation is BP), a specification from the Web Services Interoperability industry consortium (WS-I), provides interoperability guidance for core Web Services specifications such as SOAP, WSDL, and UDDI. The profile uses Web Services Description Language (WSDL) to enable the description of services as sets of endpoints operating on messages. BP complied web service means smooth internal operability (messaging, description and registry) between web services regardless of underlying implementation.

Open Source Tool for JDBC Debugging and Monitoring

2007-10-25T23:14:00.000-07:00

P6Spy is a open source framework to intercept and debug JDBC interaction with a database and optionally modify database statements. It's a wrapper of all JDBC elements ( Connection, PreparedStatement, ResultSet... ) and it has a powerful feature in order to log all informations about those interactions. P6Spy includes P6Log, an application that intercepts and logs the database statements of any application that uses JDBC.

This application is particularly useful for developers to monitor the SQL statements produced by EJB servers, enabling the developer to write code that achieves maximum efficiency on the server.

You can also use it together with spring and hibernate for performance tuning as well.

Reference List:

P6Spy

P6Spy Official Home

Integrate P6Spy with Spring

Improving Hibernate's Performance

P6Spy released: automatic logging of your database/EJB queries

XML-RPC vs JAX-RPC

2007-10-25T03:04:00.000-07:00

XML-RPC

XML-RPC is a remote procedure call protocol which uses XML to encode its calls and HTTP as a transport mechanism. XML-RPC is a very simple protocol, defining only a handful of data types and commands, and the entire description can be printed on two pages of paper. Some people still prefer XML-RPC to SOAP because of its simplicity, minimalism, and ease of use. (source wikipedia)

The original XML-RPC uses only a few data types and does not require namespaces. It achieves considerable flexibility by defining and types which can contain other values. Responses are also simple XML with as the root element with either or as the child element. The contrast with complex SOAP responses is striking.

For xml-rpc, wsdl and uddi is not involved for client to invoke the web service.

Implementations of XML-RPC:

There are five Implementations in Java list at xmlrpc.com as of December 2003. However, many of these projects appear to be inactive. The project hosted by the Apache Web Services Project is active. It looks like the Apache Web Services Project is on the right track with version 3, so keep an eye open on it if you are interested in an open-source XML-RPC solution.

JAX-RPC - The Java API for XML-based RPC

Instead of using xml-rpc protocol, JAX-RPC make used of SOAP protocol for messaging which enable web service a industry standard.

SOAP

SOAP is a protocol for exchanging XML-based messages over computer networks,normally using HTTP/HTTPS. SOAP forms the foundation layer of the Web services stack, providing a basic messaging framework so that more abstract layers can build on.

There are several different types of messaging patterns in SOAP, but by far the most common is the Remote Procedure Call (RPC) pattern, in which one network node (the client) sends a request message to another node (the server) and the server immediately sends a response message to the client. SOAP is the successor of XML-RPC, though it borrows its transport and interaction neutrality and the envelope/header/body from elsewhere, probably from WDDX. (Source: wikipedia)

WSDL is involved for client to invoke the web service when use JAX-RPC.

Is XML-RPC Still Useful?

The complexity of the JAX-RPC implementation has caused somewhat of a backlash among programmers. It seems you should consider XML-RPC over JAX-RPC if:

* Your application uses relatively simple requests
* You control both server and client sides and/or you don't need to publish a WSDL
* You want to keep client-side parsing simple (as in AJAX applications)

Architecture Chaos of Web 2.0 Application

2007-10-24T03:12:00.000-07:00

Struts, Spring, Seam, JSF, Hibernate, iBatis, Ajax and EJB, there are so many java technology, my question is which one should go for to mix and match out a wonderful J2EE Web 2.0 architecture for new project.

Model layer

Spring+Hibernate
Spring+iBatis (With skillful DBA or developers with advnaced sql skill)
EJB 3.0, forget about the EJB 2.1 (with developers expert in JDK 1.5 and annotation)

View & Controller layer

Struts1 + Ajax (zk/echo/gwt) + Tile (templating)
Spring MVC + Ajax (zk /echo/gwt) + Tile (templating)
JSF+Icefaces/Zk(AJAX)+Facelet / Tile(templating)
RichFaces / AJAX4JSF

Integration Layer

Web services
BPEL process (a collection of inter-related web services)
Enterprise Service Bus
JMS & MOM

Other than suggestion above, JBoss seam and Oracle ADF and Webcenter come with full technology stack for web 2.0 architecture.

As for which technology and plaftform should be used in the development, not only it depends on the value and benefits technology bring to the project, but available tools and people to support and accelerate the development play a part as well.

Only with good tools (development & testing) and big pool of talent people/developer, project can be delivered on time.

So, which tools should be chosen for the project development? Again, there are many open tools available in each phase of SDLC.

Someone can use a combination of tools below:

Eclipse with Checkstyle, PMD, Mylar, Subversion (Subclipse), JUnit plug-ins
Subversion version control
maven2
Jira issue tracking
Confluence wiki
Cobertura test coverage
QuickBuild Continuous Integration system
Enterprise Architect for requirements and design in UML, and document generation

But it may not fit your project and you will need to figure out your own tool set with some R&D work on it.

Here, i would like to suggest a book (not-yet-published, but coming-soon) called "Java power tools". It maintain a comprehensive open tools and I believe with this book, people can shorten the R&D work for development tools findings.

So, what is your voice for this confusion and chaos?

JAXM & Messaging

2007-10-22T20:37:00.000-07:00

JAXM Definition

JAXM, the Java API for XML Messaging, defines a lightweight Java API for composing, processing, sending, and receiving XML documents. It supports both synchronous (HTTP) and asynchronous (SMTP) messaging and “Out-of-the-box” support for ebXML messaging.

Fundamental Elements

The JAXM Service

A JAXM service consumes JAXM messages sent by a JAXM client. To develop a JAXM service, developers extend the class javax.xml.messaging.JAXMServlet and implement either the javax.xml.messaging.OnewayListener or javax.xml.messaging.ReqRespListener interface. The choice of interface depends upon whether the interaction between the client and the server is asynchronous (OnewayListener) or synchronous (ReqRespListener) in nature.

The JAXM Client

Clients exchange messages with JAXM services. JAXM clients can either interact directly with a JAXM service or go through a JAXM provider. JAXM clients that interact directly with a JAXM service can only participate in synchronous interactions. JAXM clients that use a messaging provider can participate in asynchronous as well as synchronous interactions with the JAXM service.

The JAXM Messaging Provider

A JAXM messaging provider is responsible for managing the routing of messages for a JAXM client. Besides providing a degree of separation between a JAXM client and service, a provider can also offer additional services like reliable messaging, security, and transaction support. Currently, clients that want to interact asynchronously with a JAXM service have to use a provider.

The JAXM Message

All JAXM messages conform to the SOAP 1.1 and SOAP with Attachments standards. JAXM messages also support the concept of higher-level protocols like ebXML through the concept of messaging profiles. A messaging profile defines the messaging contract between two parties. A JAXM client and service that share a message profile like ebXML can conduct business by exchanging ebXML messages.

JAXM & Messaging

* JAXM is an synchronous / asynchronous (queue-based) messaging API
* Not part of J2EE 1.4
* Uses queues on either side
* Doesn’t provide WSDL facilities
* Can be used in conjunction with JAXB to convert to objects

Difference in JAXM & JMS

The JAXM API was designed for Internet SOAP-based messaging, while the JMS API was designed as a common messaging API to be layered over existing enterprise message-oriented middleware (MOM) applications.

The delivery endpoint models differ. JAXM provides direct point-to-point messaging, while JMS uses destinations to provide a reliable asynchronous messaging capability.

JMS clients send messages to, and retrieve them from, destinations. JMS provides a model whereby an intermediary, a messaging server, makes sure a message is delivered between two applications that do not know directly about each other, but know only about the common destination.

A JAXM client is interoperable over any SOAP 1.1 compatible client (there must be an agreed-upon profile between providers, however), whereas a JMS client is interoperable only with other JMS clients over the same messaging system (or another messaging system for which the JMS provider supports a proprietary gateway).

The JMS API does not define interoperability between message providers. It does not define a common message format, only a number of message types; the clients must agree on the precise format. (It is possible to wrap a SOAP message, or any other kind of XML message, in a TextMessage and send it via JMS.)

Find out more info from JAXM FAQ

Why there is a need for JAXM?

The goal of JAXM is to provide a rich set of interfaces for document-style web services. Document-style web services enable the exchange of XML documents between two parties, as opposed to RPC-style web services, which expose software functions as web services.

JAXM-based web services can be used effectively in Such scenario:

When two parties want to exchange data using XML documents that are bounded using well-defined XML schemas instead of invoking software functions (Java objects, C procedures, etc.) exposed as RPC-style web services.

I would like to recommend you to go through a article called "Developing E-Business Interactions with JAXM" for a good understanding of How & Where to use JAXM

Implementation

Sun provides JAXM implementation, but does not support full ebXML messaging service specification, there is alternative implementation of ebMS 2.0 from www.freebxml.org.

ebXML & Message Service

Find more from my previous post here.
Find more from article called "ebXML Message Service - An In-depth Overview"

Reference List:

http://java.sun.com/webservices/jaxm
Sun JAXM tutorial
Developing E-Business Interactions with JAXM

JAX-RPC vs JAX-WS

2007-10-22T02:58:00.000-07:00

Java API for XML-Based RPC (JAX-RPC) is a Legacy Web Services Java API, it uses SOAP and HTTP to do RPCs over the network and enables building of Web services and Web applications based on the SOAP 1.1 specification, Java SE 1.4 or lower, or when rpc/encoded style must be used.

You can use the JAX-RPC programming model to develop SOAP-based web service clients and endpoints. JAX-RPC enables clients to invoke web services developed across heterogeneous platforms. Likewise, JAX-RPC web service endpoints can be invoked by heterogeneous clients. JAX-RPC requires SOAP and WSDL standards for this cross-platform interoperability.

JAX-RPC lets people develop a web service endpoint using either a Servlet or Enterprise JavaBeans (EJB) component model. The endpoint is then deployed on either the Web or EJB container, based on the corresponding component model. Endpoints are described using a Web Services Description Language (WSDL) document.(This WSDL document can be published in a public or private registry, though this is not required). A client then uses this WSDL document and invokes the web service endpoint.

JAX-RPC in J2ee 1.4 supports 4 types of stubs and invocations: static stub, dynamic proxy, Dynamic Invocation Interface (DII) and Application client.

Static Stub Client
Web service client makes a call through a stub, a local object that acts as a proxy for the remote service. Because the stub is created by wscompile at development time (as opposed to runtime), it is usually called a static stub.

Example: Invoking a Stub Client
Stub stub = (Stub) (new MyHelloService_Impl().getHelloIFPort());
stub._setProperty (javax.xml.rpc.Stub.ENDPOINT_ADDRESS_PROPERTY,endpoint_address_string);
HelloIF hello = (HelloIF)stub;
System.out.println(hello.sayHello("Duke!"));

Dynamic Proxy Client
In contrast, the client call a remote procedure through a dynamic proxy, a class that is created during runtime. Although the source code for the static stub client relies on an implementation-specific class, the code for the dynamic proxy client does not have this limitation.

Example: Dynamic Proxy
javax.xml.rpc.Service service = ServiceFactory.newInstance().createService(...);
com.example.StockQuoteProvider sqp = (com.example.StockQuoteProvider)service.getPort(portName, StockQuoteProvider.class);
float price = sqp.getLastTradePrice("ACME");

Dynamic Invocation Interface Client
With the dynamic invocation interface (DII), a client can call a remote procedure even if the signature of the remote procedure or the name of the service is unknown until runtime. In contrast to a static stub or dynamic proxy client, a DII client does not require runtime classes generated by wscompile.

Example: Dynamic Invocation Interface
javax.xml.rpc.Service service = ServiceFactory.newInstance().createService(...);
javax.xml.rpc.Call call = service.createCall(portName, "getLastTradePrice");
// This example assumes that addParameter and setReturnType methods are not required to be called
Object[] inParams = new Object[] {"ACME"};
Float quotePrice = (Float)call.invoke(inParams);

Application Client
Unlike the stand-alone clients, for an application client, because it's a J2EE component, an application client can locate a local web service by invoking the JNDI lookup method.

Example: Application Client
Context ic = new InitialContext();
MyHelloService myHelloService = (MyHelloService)
ic.lookup("java:comp/env/service/MyJAXRPCHello");
appclient.HelloIF helloPort = myHelloService.getHelloIFPort();
((Stub)helloPort)._setProperty(Stub.ENDPOINT_ADDRESS_PROPERTY,args[0]);
System.out.println(helloPort.sayHello("Jake!"));

Service Endpoint Model

JAX-RPC supports a client model for the service consumer, and a service endpoint model for the service producer.

Application-Level Interaction Modes

JAX-RPC specifies three client application interaction models:
* Synchronous request-response two-way RPC
* Asynchronous (non-blocking) request-response two-way RPC
* One-way RPC

JAX-WS 2.0 is the successor of JAX-RPC 1.1 - the Java API for XML-based Web services. If possible, JAX-WS should be used instead as it is based on the most recent industry standards.

What remains the same?

Before we itemize the differences between JAX-RPC 1.1 and JAX-WS 2.0, we should first discuss what is the same.

* JAX-WS still supports SOAP 1.1 over HTTP 1.1, so interoperability will not be affected. The same messages can still flow across the wire.

* JAX-WS still supports WSDL 1.1, so what you've learned about that specification is still useful. A WSDL 2.0 specification is nearing completion, but it was still in the works at the time that JAX-WS 2.0 was finalized.

What is different?

* SOAP 1.2
JAX-RPC and JAX-WS support SOAP 1.1. JAX-WS also supports SOAP 1.2.

* XML/HTTP
The WSDL 1.1 specification defined an HTTP binding, which is a means by which you can send XML messages over HTTP without SOAP. JAX-RPC ignored the HTTP binding. JAX-WS adds support for it.

* WS-I's Basic Profiles
JAX-RPC supports WS-I's Basic Profile (BP) version 1.0. JAX-WS supports BP 1.1. (WS-I is the Web services interoperability organization.)

* New Java features
o JAX-RPC maps to Java 1.4. JAX-WS maps to Java 5.0. JAX-WS relies on many of the features new in Java 5.0.
o Java EE 5, the successor to J2EE 1.4, adds support for JAX-WS, but it also retains support for JAX-RPC, which could be confusing to today's Web services novices.

* The data mapping model
o JAX-RPC has its own data mapping model, which covers about 90 percent of all schema types. Those that it does not cover are mapped to javax.xml.soap.SOAPElement.
o JAX-WS's data mapping model is JAXB. JAXB promises mappings for all XML schemas.

* The interface mapping model
JAX-WS's basic interface mapping model is not extensively different from JAX-RPC's; however:
o JAX-WS's model makes use of new Java 5.0 features.
o JAX-WS's model introduces asynchronous functionality.

* The dynamic programming model
o JAX-WS's dynamic client model is quite different from JAX-RPC's. Many of the changes acknowledge industry needs:
+ It introduces message-oriented functionality.
+ It introduces dynamic asynchronous functionality.
o JAX-WS also adds a dynamic server model, which JAX-RPC does not have.

* MTOM (Message Transmission Optimization Mechanism)
JAX-WS, via JAXB, adds support for MTOM, the new attachment specification. Microsoft never bought into the SOAP with Attachments specification; but it appears that everyone supports MTOM, so attachment interoperability should become a reality.

* The handler model
o The handler model has changed quite a bit from JAX-RPC to JAX-WS.
o JAX-RPC handlers rely on SAAJ 1.2. JAX-WS handlers rely on the new SAAJ 1.3 specification.

Additional reading list:

Developing Web Services at a Glance
http://java.sun.com/webservices/technologies/index.jsp#Core_Web_Services
Java API for XML-based RPC (JAX-RPC)
http://java.sun.com/developer/technicalArticles/xml/jaxrpc/

Reference:
http://java.sun.com/webservices/technologies/index.jsp#Core_Web_Services
http://www-128.ibm.com/developerworks/webservices/library/ws-tip-jaxwsrpc.html
http://www.ibm.com/developerworks/webservices/library/ws-tip-jaxwsrpc2.html

Keywords: SAAJ, JAXB, SOAP, JAX-RPC, JAX-WS

Using Oracle RAC in OAS, Weblogic and Websphere

2007-10-22T01:12:00.000-07:00

Oracle Application Server 10g R2 (OAS)

To enable the Fast Connection Failover mechanism, properties of "connectionCachingEnabled" and "fastConnectionFailoverEnabled" must be set to true on the (connection-factory) tag for an OracleDataSource object. the JDBC URL String must be set using the service name syntax. The service name specified on the connection URL is used to map the connection cache to the service.

Valid URL Usage

url="jdbc:oracle:oci:@TNS_ALIAS"

url="jdbc:oracle:oci:@(DESCRIPTION=
(LOAD_BALANCE=on)
(ADDRESS=(PROTOCOL=TCP)(HOST=host1) (PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=host2)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=service_name)))"

url="jdbc:oracle:oci:@(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=cluster_alias) (PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=service_name)))"

url = "jdbc:oracle:thin@//host:port/service_name"

url = "jdbc:oracle:thin@//cluster-alias:port/service_name"

url="jdbc:oracle:thin:@(DESCRIPTION=
(LOAD_BALANCE=on)
(ADDRESS=(PROTOCOL=TCP)(HOST=host1) (PORT=1521))
(ADDRESS=(PROTOCOL=TCP)(HOST=host2)(PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=service_name)))"

url = "jdbc:oracle:thin:@(DESCRIPTION=
(ADDRESS=(PROTOCOL=TCP)(HOST=cluster_alias) (PORT=1521))
(CONNECT_DATA=(SERVICE_NAME=service_name)))"

If a SID is specified on the URL, when Fast Connection Failover is enabled, then an exception is thrown.

For more inforamtion, find them from links below:

http://download.oracle.com/docs/cd/B31017_01/web.1013/b28958/datasrc.htm#CHDEHFHE
http://download.oracle.com/docs/cd/B31017_01/core.1013/b28939/j2ee.htm#ASTED006
http://download.oracle.com/docs/cd/B31017_01/core.1013/b28939/security.htm#CDDGFADE

Weblogic

For how to use Oracle RAC DB with Weblogic application server, find the detail information from links below:

http://edocs.bea.com/wls/docs81/jdbc/oracle_rac.html
http://edocs.bea.com/platform/docs81/db_mgmt/oracle_rac.html

Websphere

For how to use Oracle RAC DB with Websphere application server, find the detail information from links below:

http://www.ibm.com/developerworks/websphere/techjournal/0710_barghouthi/0710_barghouthi.html
http://www.ibm.com/developerworks/websphere/techjournal/0706_banerjee/0706_banerjee.html

JAXP Implementation Provider

2007-10-22T00:00:00.000-07:00

The Java API for XML Processing (JAXP) enables applications to parse (SAX/DOM), transform (XSLT), validate (Against the XML schema) and query (XPath) XML documents using an API that is independent of a particular XML processor implementation. JAXP provides a pluggability layer to enable vendors to provide their own implementations without introducing dependencies in application code. JAXP selects the parser based on system properties. You can set the properties to select a different parser than the default one.

Existing JAXP implementation include, but not limited to Resin, Xalan, Xerces and Crimson.

Xerces and Crimson is an XML parser. Xalan is an implementation of XPath and XSLT. Xalan can work with any JAXP-compliant XML parser. Resin has parser and transformer implementation for all JAXP API.

You can find the more information on each JAXP implementation from links below:

XML Parsing
http://xml.apache.org/crimson/ (sun default jaxp implementation)
http://xerces.apache.org/xerces-j/
http://www.caucho.com/resin-3.0/xml/jaxp.xtp

XMl Transformation and XPath

http://xalan.apache.org/
http://www.caucho.com/resin-3.0/xml/jaxp.xtp

Additional jaxp reading list (Past, Current and Future Roadmap)

The Evolution of JAXP
http://www.xml.com/pub/a/2005/07/06/jaxp.html?page=1

Jakarta Common Validator & Struts Validator

2007-10-19T02:38:00.000-07:00

The Commons Validator is a basic validation framework that lets you define validation rules for a JavaBean in an xml file. Validators, the validation definition, can also be defined in the xml file. An example of a validator would be defining what method and class will be called to perform the validation for a required field. Validation rules can be grouped together based on locale and a JavaBean/Form that the rules are associated with. The framework has basic support for user defined constants which can be used in some field attributes.

Here is the quick tutorial for Jakarta common validator, you can see that there is a need to define a validation rule xml file to declare the validation rules and objects & fields to be validated.

Noted, inside the xml file, the individual fields in the input data are called fields, the set of input data is collectively called a form, and a collection of forms is called a form-set.

The relationship between the elements is shown here:
Fields --> Forms --> FormSet

Struts Validator is built on Jakarta Commons validator and other common packages. It split the rule defintion xml file into two, one (validation-rules.xml) is for rule definitions, another (validation.xml) is for which and what object and fields to be validated. Struts validator is a web form validator and support both client and server side validation.

Here is a tutorial for how to use the Struts Validator.

EJB Reference Definiton and Lookup in Oracle Application Server 10g

2007-10-16T21:33:00.000-07:00

EJB Reference Definition

There are two way for EJB reference definition in Oracle AS:

One is define the (ejb-ref-name) together with (ejb-link) within the (ejb-ref) element. Configure (ejb-ref-name) with a logical name and configure (ejb-link) with the actual name of the target bean as shown below:

Example: Configuring ejb-ref-name with a Logical Name Resolved by ejb-link

(ejb-ref)
(ejb-ref-name)ejb/nextVal(/ejb-ref-name)
(ejb-ref-type)Session(/ejb-ref-type)
(home)myBeans.BeanAHome(/home)
(remote)myBeans.BeanA(/remote)
(ejb-link)myBeans/BeanA(/ejb-link)
(/ejb-ref)

This option provides indirection that offers assembly and deployment flexibility.

Another way is to Configure (ejb-ref-name) with a logical name and in the orion-ejb-jar.xml deployment descriptor, define an (ejb-ref-mapping) element that maps the logical name to the actual name of the target bean as shown below:

Example: Configuring ejb-ref-name with a Logical Name Resolved by ejb-ref-mapping

(ejb-ref)
(ejb-ref-name)ejb/nextVal(/ejb-ref-name)
(ejb-ref-type)Session(/ejb-ref-type)
(home)myBeans.BeanAHome(/home)
(remote)myBeans.BeanA(/remote)
(/ejb-ref)

Example: Mapping Logical Name to Actual Name with ejb-ref-mapping

(ejb-ref-mapping name="ejb/nextVal" location="myBeans/BeanA"/)

This option provides indirection that offers the most assembly and deployment flexibility.

EJB Reference Lookup

To lookup EJB in the OAS, there are two ways too. One is using the ejb-ref (logical name in the web.xml/ejb-jar.xml) and second way is using the real location (jndi name in the server).

For sample code snippet, refer to links below:

http://download.oracle.com/docs/cd/B25221_04/web.1013/b14428/servjndi.htm#CHDGBAFJ
http://download.oracle.com/docs/cd/B25221_04/web.1013/b14427/jndi.htm#i1084614
http://download.oracle.com/docs/cd/B25221_04/web.1013/b14427/jndi.htm#CIHGAHFF

Oracle Application Server 10g How-To's

2007-10-16T21:01:00.001-07:00

This link provides a high-level overview of a feature along with relevant code excerpts to illustrate how the feature can be used.

Infrastructure
How-To Use Application Properties
How-To Register MBeans on Container Startup
How-To Create Simple JMX MBeans Mar-2006
How-To Swap Oracle JDBC Drivers
How-To Swap XML Parsers
How-To Configure and Use Job Scheduler with database persistence
How-To Configure and Use Job Scheduler with JMS persistence
How-To Configure and Use JCA 1.5 Resource Adapters
How-To Configure and Use Data Sources
How-To Configure and Use JNDI
How-To Configure and Use Proxy-authentication with Data Sources
How to Configure and use Active MQ JCA 1.5 Resource Adapter with OC4J 10 g (10.1.3)
How-To Use ORMI/SSL with OC4J
How-To Propagate a transaction context between OC4J instances
How-to Stop an MDB via JMX (Blog)
How-to Establish a Remote connections to OC4J with JConsole (Blog)

Servlets and JSP / Web Container
How-To Use Dependency Injection in a Web Module
How-To Use Servlet 2.4 Features
How-To Use HTTPClient in OC4J 10g

Web Services
How-To Use MTOM to exchange binary data
How-To Develop a Web Service From an Existing Java Class
How-To Develop a Web Service From a WSDL Document
How-To Generate a Web service using Web service Metadata Annotations (JSR 181)
How -To Use WS Security between OracleAS Web Services and Microsoft .Net
How to Use JDeveloper 10.1.3 to Secure and Test a Web Service
How-to Share State of Stateful Web Services Between Clients (Blog)
How to use SOAP Compression using JAX-RPC, on OC4J (Blog)
How to use SSL with OracleAS Web Services (Blog)

Spring Framework
Using the New Spring JTA Transaction Manager
Use EJB 3.0 and Java Persistence API with Spring in OC4J
Spring-TopLink Pet Clinic Demo

Messaging (JMS)
How-To Use the OracleAS JMS Router
How to Configure and Use Oracle's Generic JMS Resource Adapter
With IBM WebSphere MQ JMS
With Tibco Enterprise for JMS
With SonicMQ JMS
With OEMS JMS (In-Memory and File-Based)
With OEMS JMS (Database)
How-to Create as Simple MDB with Oracle Database JMS Provider (Blog)
How-to Propagate JMS Messages to Multiple Targets (Blog)
How-to use OracleAS JMS, Hermes and MBeans (Blog)

J2EE Security / JAAS
How-To Integrate a Custom JAAS Login Module in OracleAS
How-To Reset oc4jadmin password (Blog)

Oracle Rules Engine
How-To Build a Rules Enabled Java Application
How-To build a Rules Enabled Application using XML facts

Additional Resources
Tutorial for Java EE Developers (10.1.3.1.0)
How to use SASH Starter 2.0 with OracleAS 10g R3
EJB 3.0 How-To's
Oracle TopLink How-To's
Oracle JDeveloper How-To's
How-To Use Eclipse with OC4J 10g (Eclipse Web Tools Project Page)

Where did System.out | System.err statements print in Oracle AS 10g (10.1.3)?

2007-10-05T00:36:00.000-07:00

By default, Sys.out statements are printed in the $Oracle_Home/opmn/logs/_default_island_# folder. But we can change the location by modify the opmn.xml file.

To specify other out file location:

We use "-out [file]" option in the oc4j startup parameter list to Specifies a file to route the standard output to. The specified file contains messages that are printed to System.out, as well as the messages sent to output through the servlet logging interface. If not specified, all output is written to standard out.

Note that in an OPMN-managed configuration, the file will be generated within an instance_default_group_1 directory appended to the path specified.

For example, if you specify the following in opmn.xml:

The mylog.log file will actually be generated in:

/mypath/instanceName_default_group_1/mylog.log.

To specify other err file location:

We use "-err [file]" option in the oc4j startup parameter list to specify a file to route standard error output to. The file contains messages that are printed to System.err. If not specified, all errors are written to standard error.

Note that in an OPMN-managed configuration, the file will be generated within an instance_default_group_1 directory appended to the path specified. See the -out description above for details.

See "Managing stdout/stderr Log Files" for additional system properties that can be set to manage stdout files.

TIBCO Rendezvous vs SonicMQ

2007-10-04T01:05:00.000-07:00

This article is compiled from source available in the WWW, not my own opinion and only used for reference and information purpose.

TIBCO Rendezvous

TIBCO has been one of the leading providers of EAI since its establishment 20 years ago and TIB/RV is one of the most widely used messaging middleware in enterprises.

TIB/RV is implemented in C and based on a distributed architecture. An installation of TIB/RV resides on each host on the network.

Messaging can be delivered in point-to-point or publish/subscribe, synchronously or asynchronously, locally delivered or sent via WAN or the Internet. Rendezvous messages are self-describing and platform independent.

TIB/RV is composed of three main components:
- RV Daemon (RVD) responsible for the delivery of messages within a LAN.
- RV Agent (RVA)
- RV Routing Daemon (RVRD)

In TIB/RV, point-to-point messages sending model is fairly similar to publish/subscribe, The RV Sender program passes the message and destination topic to RVD. RVD then broadcasts this message using User Data Packet (UDP) to the entire network. All subscribing computers with RVDs on the network will receive this message. RVD will filter the messages which non-subscribers will not be notified of the message. Therefore only subscriber programs to the particular topic will get the messages.

SonicMQ

SonicMQ is JMS implementation, and has outstanding performances competitive with existing MOM technologies, such as IBM MQSeries. SonicMQ is written in 100% pure Java, supports XML messaging, and HTTP tunnelling to allow SonicMQ to work over the Internet. The underlying mechanism of SonicMQ is its “broker” that facilitates the movement of messages across the network.

The communication protocols that can be used with SonicMQ include TCP, HTTP and SSL. Since it uses common Internet protocol, SonicMQ can extend its deployment to the Internet. It also provides bridges to many other popular MOMs that allow messages to be sent and received between SonicMQ and other MOMs.

There are three types of configurations a user can choose from:
- Single-broker Configuration: Under this configuration, there is one broker which is being shared across a few nodes.
- Multi-broker Clusters
- Multi-node Configurations

The broker is the most important underlying implementation of SonicMQ. It is responsible for delivering and acquiring of messages within a LAN environment. It is a client-server model, where many clients connect to a single broker. The connection can be via TCP (for LAN), SSL (for security encryption), or even HTTP (to connect to external entities).

The downside with single broker configuration is that scalability is limited by the capabilities of the node machine. Also the system is dependent on the single broker machine (node), hence leading to a bottleneck of the system at the node. The whole system may collapse if the node goes down. To solve this problem a multi-broker cluster must be used.

There exist a benchmark report for SonicMQ by Progress Software. SonicMQ showed outstanding performances compared to IBM MQSeries and Fiorano FioranoMQ, (both are JMS implementations) under WinNT platform.

TIB/RV vs SonicMQ

There is a benchmarking result shows that TIB/RV has exceptional performance compared to SonicMQ.

They are as follows:
- High publish/subscribe and point-to-point send/receive rates
- High scalability:
- Publishing rate not affected by introducing more receivers;
- Subscriber rate increases as more subscribers are introduced.
- Low memory and CPU consumption

The only major downside of TIB/RV is that when there are very few receivers in the network, it could flood the network with many unnecessary UDP packets, introducing congestions.

TIB/RV is using multicast-based publish/subscribe messaging, which is an excellent solution for near-real-time message dissemination when 1 to 'very-many' delivery capabilities matter. It is better for application with high volume, high performance, acceptance of unreliability (nature of UDP), 1 to very many.

However, in most EAI cases the opposite requirements are true: ‘Cardinality’ of message delivery is 1-1, 1-2; 1 to-very-many is a rare case. With exception of ‘consolidated application’ integration model (near real time request reply with timeout heuristics), reliability of message delivery takes priority over performance.

Reference List

Benchmarking Message-Oriented Middleware TIB/RV vs SonicMQ
Tibco RV vs Tibco EMS

Hands-on Tutorial of Oracle BI EE

2007-10-03T02:34:00.000-07:00

1. BI EE Installation http://www.oracle.com/technology/obe/obe_bi/bi_ee_1013/install/installoraclebee.htm

2. Create A Repository
http://www.oracle.com/technology/obe/obe_bi/bi_ee_1013/bi_admin/biadmin.html

3. Create A DashBoard http://www.oracle.com/technology/obe/obe_bi/bi_ee_1013/saw/saw.html

4. Integration with BI Publisher http://www.oracle.com/technology/obe/obe_bi/bipub/bip_biee/bip_biee.htm

OU Reference
http://ouweb.us.oracle.com/database/oracle_business_inteligence_series_replays.html

BI Primer

2007-10-03T01:57:00.000-07:00

Recently, i am looking at the Oracle Business Intelligence (BI) solution and preparing a technical sharing session.

When I dive into the BI, i found it is a damn big topic and a lot of acronym and technical term need to be clear before get to understand it.

Information here is compiled from multiple source available from the internet to clear those prerequisite.

What is BI ?

Business intelligence (BI) is a business management term, which refers to applications and technologies that are used to gather, provide access to, and analyze data and information about company operations.

BI systems can help companies have a more comprehensive knowledge of the factors affecting their business, such as metrics on sales, production, internal operations, and they can help companies to make better business decisions.

BI software and applications includes a range of tools. Some BI applications are used to analyze performance, projects, or internal operations. Other BI applications are used to store and analyze data, such as Data mining (DM), Data Farming, and Data warehouses; Decision Support Systems (DSS) and Forecasting. Other BI applications are used to analyze or manage the "human" side of businesses, such as Customer Relationship Management (CRM) and Marketing tools and Human Resources applications.

BI often uses key performance indicators (KPIs) to assess the present state of business and to prescribe a course of action. The KPI methodology was further expanded with the Chief Performance Officer methodology which incorporated KPIs and root cause analysis into a single methodology.

What is OLTP ?

OLTP stand for Online Transaction Processing. This is a standard, normalized database structure. OLTP is designed for transactions, which means that inserts, updates, and deletes must be fast. Imagine a call center that takes orders. Call takers are continually taking calls and entering orders that may contain numerous items. Each order and each item must be inserted into a database. Since the performance of the database is critical, we want to maximize the speed of inserts (and updates and deletes). To maximize performance, we typically try to hold as few records in the database as possible. OLTP is typically for data entry and retrieval transaction processing.

What us OLAP ?

Online Analytical Processing is an approach to quickly providing answers to analytical queries that are multidimensional in nature. OLAP is part of the broader category BI, which also includes Extract transform load (ETL), relational reporting and data mining. The typical applications of OLAP are in business reporting for sales, marketing, management reporting, business process management (BPM), budgeting and forecasting, financial reporting and similar areas. The term OLAP was created as a slight modification of the traditional database term OLTP (Online Transaction Processing).

The output of an OLAP query is typically displayed in a matrix (or pivot) format. The dimensions form the row and column of the matrix; the measures, the values.

What is an OLAP cube ?

The term "cube" comes from the geometric object that has three dimensions. OLAP cubes can have many more dimensions than 3, but the term continues to apply.

In database theory, an OLAP cube is an abstract representation of a projection of an RDBMS relation. It is an arrangement of data in arrays (alternatively called cubes) to allow fast analysis. The arrangement of data into cubes avoids a limitation of relational databases which are not well suited for near instantaneous analysis of large amounts of data. OLAP cubes can be thought of as extensions to the two-dimensional array of a spreadsheet. Because there can be more than three dimensions in an OLAP system the term hypercube is sometimes used. The commercial OLAP products have different methods of creating the cubes and hypercubes and of linking cubes and hypercubes.

What is Measure, Dimension & Hierarchy?

While most databases designed for OLTP such as those used in claims processing are designed for efficiency in data storage, OLAP cubes are designed for efficiency in data retrieval.

Regular relational databases treat all data into the database similarly, however OLAP cubes categorize data into "dimensions" and "measures".

Measures represent items that are counted, summarized or aggregated, such as costs or units of service.

A company might wish to analyse some financial data by product, by time-period, by city, by type of revenue and cost, and by comparing actual data with a budget. These additional methods of analysing the data are known as dimensions.

Dimensions are variables by which measures are summarized, such as hospitals, physicians, or dates of service. This organization of data greatly facilitates the ability to formulate data requests based on real-life situations.

Each of the elements of a dimension could be summarised using a hierarchy. For example May 2005 could be summarised into Second Quarter 2005 which in turn would be summarised in the Year 2005. Similarly the cities could be summarised into regions, countries and then global regions; products could be summarised into larger categories; and cost headings could be grouped into types of expenditure. Conversely the analyst could start at a highly summarised level such as the total difference between the actual results and the budget and drill down into the cube to discover which locations, products and periods had produced this difference.

In addition, many of queries that could be posed to the data are "pre-aggregated" in the database such that the answers have already been precalculated and can be reported without delay.

What is Oracle OLAP Cube?

Oracle OLAP Cube requires a specific set of materialized views to map to a star schema. A materialized view is a replica of a target master from a single point in time. The master can be either a master table at a master site or a master materialized view at a materialized view site. ( more about 1 | 2).

Why are OLAP cubes important ?

Before OLAP technology was well developed, data had to be extracted from databases using "queries".

This meant that the analyst had to structure a request to the database for the information desired, and then submitted this query to the database server. That server would processing query and return the results.

Depending on the size of the database and the data requested, this query could take minutes or hours to complete.

In this sense, the "online" aspect of this type of reporting is questionable. OLAP cubes are fundamentally different in that they "pre-aggregate" the data used to answer many of queries that are anticipated.

What is star schema?

The star schema (sometimes referenced as star join schema) is the simplest style of data warehouse schema, consisting of a few "fact tables" (possibly only one, justifying the name) referencing any number of "dimension tables". The "facts" that the data warehouse helps analyze are classified along different "dimensions": the fact tables hold the main data, while the usually smaller dimension tables describe each value of a dimension and can be joined to fact tables as needed.

Dimension tables have a simple primary key, while fact tables have a compound primary key consisting of the aggregate of relevant dimension keys.

Facts and Dimensions

When we talk about the way we want to look at data, we usually want to see some sort of aggregated data. These data are called measures. These measures are numeric values that are measurable and additive. For example, our sales dollars are a perfect measure.
Just tracking measures isn't enough, however. We need to look at our measures using those "by conditions". These "by conditions" are called dimensions.

Therefore, in designing a star schema, our first order of business is usually to determine what we want to see (our measures) and how we want to see it (our dimensions).

Building the Fact Table

The Fact Table holds our measures, or facts. The measures are numeric and additive across some or all of the dimensions. For example, sales are numeric and we can look at total sales for a product, or category, and we can look at total sales by any time period. The sales figures are valid no matter how we slice the data.

While the dimension tables are short and fat, the fact tables are generally long and skinny. They are long because they can hold the number of records represented by the product of the counts in all the dimension tables.

What is data mining?

Data mining has been defined as "the nontrivial extraction of implicit, previously unknown, and potentially useful information from data" and "the science of extracting useful information from large data sets or databases".

Data mining involves sorting through large amounts of data and picking out relevant information. It is usually used by Business intelligence organizations, and financial analysts, but is increasingly used in the sciences to extract information from the enormous data sets generated by modern experimental and observational methods.

What is oracle OLAP Solution?

Oracle OLAP solution has Oracle DB OLAP Option and Hyperion Essbase.

Oracle Database OLAP is optimal for accelerating query performance and
adding analytic content for SQL-based BI Tools on Oracle Database, whereas oracle’s Hyperion Essbase is ideal for performance management applications to be the de-facto world leader in planning, budgeting applications etc.

Both the OLAP Option and Essbase share certain core capabilities which are fast query performance, fast incremental update of data sets, rich calculation capabilities and dimensional calculation and query models.

Oracle OLAP—an option to Oracle Database 11g Enterprise Edition—is a calculation engine that supports the entire spectrum of advanced analytical applications. It extends the analytic capabilities of the Oracle database by providing new multidimensional datatypes, a multidimensional calculation engine, and a framework to build OLAP applications using SQL, PL/SQL, and Java. (Know more about oracle OLAP option)

Database vs. Data Warehouse

The primary difference betwen you application database and a data warehouse is that while the former is designed (and optimized) to record , the latter has to be designed (and optimized) to respond to analysis questions that are critical for your business. Application databases are OLTP (On-Line Transaction Processing) systems where every transation has to be recorded, and super-fast at that.This system is write-optimized, and you shouldn?¡¥t crib if your analysis query (read operation) takes a lot of time on such a system.

A Data Warehouse (DW) on the other end, is a database (yes, you are right, it?¡¥s a database) that is designed for facilitating querying and analysis. Often designed as OLAP (On-Line Analytical Processing) systems, these databases contain read-only data that can be queried and analysed far more efficiently as compared to your regular OLTP application databases. In this sense an OLAP system is designed to be read-optimized.

Creation of a DW leads to a direct increase in quality of analyses as the table structures are simpler (you keep only the needed information in simpler tables), standardized (well documented table structures), and denormalized (to reduce the linkages between tables and the corresponding complexity of queries). A DW drastically reduces the ??cost-per-analysis?¡¥ and thus permits more analysis per FTE. Having a well-designed DW is the foundation successful BI/Analytics initiatives are built upon (original post).

Data Warehouse vs Data Mart

Data Warehouses and Data Marts are conceptually different in scope. However, they are built using the exact same methods and procedures.

A Data Mart is a specific, subject oriented, repository of data designed to answer specific questions for a specific set of users. So an organization could have multiple data marts serving the needs of marketing, sales, operations, collections, etc. A data mart usually is organized as one dimensional model as a star-schema (OLAP cube) made of a fact table and multiple dimension tables.

In contrast, a Data Warehouse (DW) is a single organizational repository of enterprise wide data across many or all subject areas. The Data Warehouse is the authoritative repository of all the fact and dimension data (that is also available in the data marts) at an atomic level.

A data warehouse (or mart) is way of storing data for later retrieval. This retrieval is almost always used to support decision-making in the organization. That is why many data warehouses are considered to be DSS (Decision-Support Systems). oth a data warehouse and a data mart are storage mechanisms for read-only, historical, aggregated data.

As technology improved (lower cost for more performance) and user requirements increased (faster data load cycle times and more features), data warehouses have evolved through several fundamental stages:

* Offline Operational Databases ?a Data warehouses in this initial stage are developed by simply copying the database of an operational system to an off-line server where the processing load of reporting does not impact on the operational system's performance.
* Offline Data Warehouse ?a Data warehouses in this stage of evolution are updated on a regular time cycle (usually daily, weekly or monthly) from the operational systems and the data is stored in an integrated reporting-oriented data structure
* Real Time Data Warehouse ?a Data warehouses at this stage are updated on a transaction or event basis, every time an operational system performs a transaction (e.g. an order or a delivery or a booking etc.)
* Integrated Data Warehouse ?a Data warehouses at this stage are used to generate activity or transactions that are passed back into the operational systems for use in the daily activity of the organization.

The term data warehouse architecture is primarily used today to describe the overall structure of a Business Intelligence system. Other historical terms include decision support systems (DSS), management information systems (MIS), and others (original post).

List of reference i used to compile this primer:

Introduction to OLAP
http://www.dwreview.com/OLAP/Introduction_OLAP.html

Data Warehousing Overview
http://www.dwreview.com/DW_Overview.html

Designing OLAP Solutions
http://www.dwreview.com/OLAP/OLAP_Comparison.html

OLAP architectures
http://www.olapreport.com/Architectures.htm

Designing the Star Schema Database By Craig Utley
http://www.ciobriefings.com/whitepapers/StarSchema.asp

Open Source Analytics / Open source BI by Nishith
http://opensourceanalytics.com/

Discussion of designing OLAP Cubes
http://databaseanswers.org/designing_olap_cubes.htm

Vector graphics Vs Raster graphics

2007-10-03T01:10:00.000-07:00

Source: wikimedia.org
Vector graphics (also called geometric modeling or object-oriented graphics) is the use of geometrical primitives such as points, lines, curves, and polygons, which are all based upon mathematical equations to represent images in computer graphics.

It is used in contrast to the term raster graphics, which is the representation of images as a collection of pixels, and used as the sole graphic type for actual photographic images. To know more about vector | raster

Oracle Locator Vs Oracle Spatial

2007-10-03T00:39:00.000-07:00

To get a quick idea of what difference between oracle locator and spatial are, i put them down here.

Features included in Oracle Locator:

* Exactly the same code base as Oracle Spatial for included functions - you won't get a performance degrade by using Oracle Locator, and features behave in the same manner in Oracle Locator and Oracle Spatial;
* Full support to store spatial data types;
* Standard SQL query access to spatial data;
* All indexing types are available - including R-Tree and partition support;
* No limits on the size and complexity of data stored;
* Most of the common spatial operations, such as "find all customers in this region" and "what shops are within 100 metres of this school", are available;
* Data compatibility between Oracle Spatial and Locator, so if you do decide to upgrade, all your data remains valid; and
* Support for multiple coordinate systems.

Oracle Spatial provides the following functions which are not included in Oracle Locator:

* Linear referenced features e.g. roads which store the distance along the road;
* Network models, such as road networks, allowing rapid calculation of shortest paths;
* Topological relationships can be stored, such as this property boundary touches this neighbouring boundary. Generally used by data producers wishing to edit high integrity spatial data;
* Storage of images such as satellite and aerial photography;
* A geocoder that translates address data to coordinate; and
* Some more complex spatial functions, mainly for analysis and data aggregation.

Find the original article here.

There a list of relevant readings from where you can go further.

Oracle Spatial - Wikipedia
Oracle Spatial Documents - Documentation
Oracle Locator and Spatial Option - Datasheet
Oracle Locator and Spatial FAQ - FAQ
Oracle XE , Locator & Geospatial Information Systems - Interview

Tech Forum & Workshop in Kuala Lumpur, Singapore & Indonesia

2007-09-28T01:21:00.000-07:00

I have been staying in KL for the last 4 days for a tech forum and hands-on workshop hosted in KL.

This forum explains how ISVs can build their applications using oracle technology and grow their business with market leaders (Oracle & Sun). It also covers the new features, benefit and how to use technical resources available exclusively to our ISVs.

In the workshop, Oracle (My team) will assist ISVs attendees migrating their application from other databases or middleware to Oracle technology.

Here is the event agenda

Day 1:

09:00 AM Registration
09:30 AM Welcome note and agenda
09:45 AM Tech Talk By Sun (Open Solaris and Virtualization)
10:45 AM Morning Break
11:00 AM Tech Talk By Oracle ( Migration Methodology and Best Practices)
12:00 PM Lunch
01:30 PM Migration workshop
03:30 PM Afternoon Break
03:45 PM Migration workshop
06:00 OM Close

Day 2:

09:00 AM Continue Migration workshop
10:45 AM Morning Break
12:00 PM Lunch
01:30 PM Continue Migration workshop
03:30 PM Afternoon Break
03:45 PM Continue Migration workshop
06:00 OM Closing Ceremony

I reached KL two days earlier before the event open. For the first two days of my staying, I am quite busy in preparing the user environment like work stations, servers (from Sun) and network setup as this is the first time for our Malaysia partner (co-organizer) hosting this kind of event. At the last 2 days, i gave the technical talk about migration methodology and help ISVs solve their issues during the hands-on migration session.

In my speech, i covered the migration methodology, approaches and things they need to consider for the DB migration , J2EE & Application Server (AS) migration and Oracle Forms migration. Some migration tools from oracle have been introduced to our ISVs, with which migration can be automated & simplified and duration of migration can be shorten. Demonstration of how to use those tools is also given after the speech.

Here is the material for the migration, you can dig out more from the links i am giving in the ppt.

There are actually some photos taken by our partner on that day and I am in the midst of geting them and upload here. So, keep tuned for the photos.

Migration workshop in Kuala Lumpur

Migration workshop in Kuala Lumpur

Tech Forum in Singapore

Sun Talk about Virtualization in Singapore

Tech Forum in Indonesia

Talk about DB Migration in Indonesia

Talk about DB Migration in Indonesia

Real life architectures

2007-09-27T23:53:00.000-07:00

Want to know how the architectures in real world which scale to millions of users looks like? Visit highscalability.com, You can find these information for google, amazon, youtube and flickr.

Google Architecture
http://highscalability.com/google-architecture

Amazon Architecture
http://highscalability.com/amazon-architecture

YouTube Architecture
http://highscalability.com/youtube-architecture

Flickr Architecture
http://highscalability.com/flickr-architecture

There are more real life architectures and you can find it here.

Highscalability.com is really a great work and i had added it to my rss feed list.

A portfolio of Oracle content management product

2007-08-02T01:52:00.000-07:00

Oracle content management product portfolio includes Oracle Universal Content Management, Oracle Universal Records Management, Oracle Imaging and Process Management, Oracle Information Rights Management and Oracle Content Database.

- Oracle Universal Content Management: provides a unified application for delivering document management, web content management, digital asset management, and retention management on a single platform. more ...

- Oracle Universal Records Management: provides the most comprehensive DoD 5015.2 compliant records management system, enabling customers to apply records and retention management policies and practices to content in remote multiple repositories and applications including file systems, content management systems, and email archives as well as addressing the management of physical records in the same system. more ...

- Oracle Imaging and Process Management: provides image scanning and business process management capabilities for enterprise applications such as accounts payable and time and expense activities and includes support for PeopleSoft, JD Edwards, and Oracle eBusiness Suite. Its complete imaging management--from creation to archiving -includes certified integrations with Oracle Applications (including the Oracle E-Business Suite, JD Edwards, and PeopleSoft Enterprise) as well as other third-party business applications. more ...

- Oracle Information Rights Management: provides policy-based security and retention management on documents when they leave a repository. It uses encryption to "seal" documents to secure and track them everywhere they're stored and used-enablin organizations to define fine-grained policies for viewing, editing, printing, and copying sensitive information, ensuring that only authorized users have access to secured content. And, when the content or user is no longer valid, rights can be easily revoked. more ...

- Oracle Content Database Suite: an enterprise content infrastructure, that is highly scalable for global deployments and is based on the industry leading and highly scalable Oracle database. Supports file server and archive consolidation as well as a content repository for enabling and building content-centric applications. more ...

First three products are bundled together and known as Enterprise Content Management Suite.

There is a short story about what oracle did recently in the cm development, check it out if you are interested.

Build Oracle RAC 10g Release 2 Cluster on Linux and FireWire

2007-07-27T02:15:00.000-07:00

Note, before going through this article, you better clear some definition like SAN / NAS / Fibre Channel / FireWire / iSCSI.

One of the most efficient ways to become familiar with Oracle Real Application Clusters (RAC) 10g technology is to have access to an actual Oracle RAC 10g cluster. There's no better way to understand its benefits—including fault tolerance, security, load balancing, and scalability—than to experience them directly.

Find the original article here.

The Open Group Architecture Framework

2007-07-25T19:18:00.000-07:00

The Open Group Architecture Framework (TOGAF) is a comprehensive architecture framework and methodology from open group which enables the design, evaluation and implementation of the right architecture for an enterprise.

There is an Architecture Review Checklist which you can use for architecture review for your project.

Find out more from here.

Journey of learning and thinking

2007-07-25T01:14:00.000-07:00

Recently, i got certified as an enterprise architect from Sun. The Journey of making it happen is really painful and fruitful. Saying it painful is that i burned all my weekend for it and nobody around (people like BA) i can discuss or clarify (what i did in real project), which put me into my own assumption and a bad assumption may lead me to a deadlock state easily. What i did for this case was either to google it from internet or post my questions to some forums, by which i will be in the long-waiting idle state until someone come and share with me (very long waiting time make me sad and frustrated). From the other side, i should say it fruitful as it did build up my logical mindset for viewing, decomposing and architecting an application system.

The whole process took me almost one year, it is truly a journey of learning, thinking, questioning, digesting and accumulating instead of 'hacking' as what i did to be a certified programmer long time ago. For the latter, i have learned all programming skill in school for years, don't wanna repeat the same thing again, but clear it in short time by doing mock exams (my way of hacking). For this SCEA, it is really a new milestone for me and something more exciting. Hacking of it may works for me and take less time, but not good for my own as i wanna self-train toward to a higher stage not for the seek of having one more cert in my profile.

BPEL4People Specifications Integrate Human Interactions Into Business Process

2007-07-22T19:58:00.001-07:00

A group of six technology vendors, including Active Endpoints, Adobe, BEA Systems, IBM, Oracle, and SAP AG, has announced the publication of 'BPEL4People' specifications, which define an approach for integrating human interactions using Web Services Business Process Execution Language (WS-BPEL) 2.0.

BPEL4People extends the capabilities of WS-BPEL to support a broad range of human interaction patterns, allowing for expanded modeling of business processes within the WS-BPEL language.

Specification Extracts

BPEL4People is comprised of two specifications including:

- WS-BPEL Extension for People, which layers features on top of WS-BPEL to describe human tasks as activities that may be incorporated as first-class components in WS-BPEL process definitions.

- Web Services Human Task introduces the definition of stand-alone human tasks, including the properties, behavior and operations used to manipulate them. Capabilities provided by Web Services Human Task may be utilized by Web services-based applications beyond WS-BPEL processes.

The WS-BPEL Extension for People specification introduces a set of elements which extend the standard BPEL elements and enable the modeling of human interactions, which may range from simple approvals to complex scenarios such as separation of duties, and interactions involving ad-hoc data. The specification introduces the people activity as a new type of basic activity which enables the specification of human interaction in processes in a more direct way. The implementation of a people activity could be an inline task or a standalone human task defined in the WS-HumanTask specification.

The WS-HumanTask language introduces a grammar for describing human tasks and notifications. Both design time aspects, such as task properties and notification properties, and runtime aspects, such as task states and events triggering transitions between states are covered by the language. Finally, it introduces a programming interface which can be used by applications involved in the life cycle of a task to query task properties, execute the task, or complete the task. This interface helps to achieve interoperability between these applications and the task infrastructure when they come from different vendors.

Visit original post for more.

New features and enhancements in Java 5.0 & 6.0

2007-07-22T19:57:00.001-07:00

Following are the changes and enhancements at the language level for Java 5.0

Generics
This long-awaited enhancement to the type system allows a type or method to operate on objects of various types while providing compile-time type safety. It adds compile-time type safety to the Collections Framework and eliminates the drudgery of casting.

Enhanced for Loop
This new language construct eliminates the drudgery and error-proneness of iterators and index variables when iterating over collections and arrays

Autoboxing/Unboxing
This facility eliminates the drudgery of manual conversion between primitive types (such as int) and wrapper types (such as Integer).

Typesafe Enums
This flexible object-oriented enumerated type facility allows you to create enumerated types with arbitrary methods and fields. It provides all the benefits of the Typesafe Enum pattern ("Effective Java," Item 21) without the verbosity and the error-proneness.

Varargs
This facility eliminates the need for manually boxing up argument lists into an array when invoking methods that accept variable-length argument lists.

Static Import
This facility lets you avoid qualifying static members with class names without the shortcomings of the "Constant Interface antipattern."

Metadata (Annotations)
This language feature lets you avoid writing boilerplate code under many circumstances by enabling tools to generate it from annotations in the source code. This leads to a "declarative" programming style where the programmer says what should be done and tools emit the code to do it. Also it eliminates the need for maintaining "side files" that must be kept up to date with changes in source files. Instead the information can be maintained in the source file.

For more features and enhancements in Java 5.0, visit Sun's official page.

For Java 6.0 (a.k.a Mustang), no significant changes at the language level, but comes with a bunch of enhancements in the other areas like Core, XML and Desktop. Most of the features are applicable both to J2SE and J2EE Platforms. New features and enhancements are encapsulated in the form of a Java Specification Request (JSR). Below is list of features we will use frequently (IMHO) in the near future.

Pluggable Annotation Processing API (JSR 269)
Common Annotations (JSR 250)
Java API for XML Based Web Services - 2.0 (JSR 224)
JAXB 2.0 (JSR 222)
Web Services Metadata (JSR 181)
Streaming API for XML (JSR 173)
JDBC 4.0 (JSR 221)
Scripting in the Java Platform (JSR 223)

For the explanation and example of each, you can go through the article (part 1 and 2) written by Shunmuga Raja. For comprehensive features and enhancements in Java 6.0, visit Sun official page.

Does JavaFX spell the end Of AJAX?

2007-07-22T19:56:00.002-07:00

You know all that AJAX code you've been writing and tearing your hair out over as you attempt to get the JavaScript working in both Internet Explorer and Firefox? Yeah, that AJAX code. It's all going to be useless real soon. Find out more from its original post here.

Note, this is not my view. Every language always has its strength and weakness and it is just a matter of application in the right place and at right time.

JavaFX Script

2007-07-22T19:56:00.001-07:00

Furthering the promise of “write once, run anywhere”, Sun release a new product family, called Java FX, which is enabling developers to build compelling, rich user interfaces that leverage Java for a wide spectrum of devices. JavaFX initially is comprised of JavaFX Script and JavaFX Mobile

JavaFX Script will run on any JavaSE technology-based platform including all of the upcoming JavaFX software systems for mobile handsets, TVs and other embedded applications from automobiles to game systems.

JavaFX Script is unique in providing close integration with Java components that run on the server or the client, resulting in a richer end-to-end experience. JavaFX Script brings together a simple and intuitive language design, requiring less coding and providing fast development cycles with a ubiquitous runtime platform and an open source program for innovation by developers worldwide.

Sun plans to make JavaFX Script available under an open source license and release the early alpha version of JavaFX Script at openjfx.org on Java.net.

You can check out the demo and tutorial.

(Notes: you may have problem when viewing the demo within oracle network, change it to wifi/public network).

Advanced Queues and Streams in Plain English & How-To

2007-07-22T19:55:00.001-07:00

Find the original post here.

As of Oracle release 10.1, AQ is integrated into Oracle Streams, and is called "Oracle Streams AQ".

What is AQ? Advanced Queues, or AQ, is Oracle's messaging solution. AQ provides a persistent (or non-persistent for specialty apps) queue mechanism that can guarantee delivery of a message. It has interfaces to PL/SQL, OCI and Java. It's Oracle's answer to IBM's MQ Series.

A message can be an XML document, a set of fields, an array of data and just about anything else you can think of.

AQ works on a publish/subscribe model. That means that someone (a publisher puts a message on the queue and someone else (a subscriber) takes the message off. A queue can have multiple subscribers. Technically it can have multiple publishers but I haven't worked with that configuration and I'm not sure what the usefulness of that is. I think I would prefer multiple queues, one for each publisher.

What's AQ good for? What does it do?

One example would be replication. In Oracle Advanced Replication, AQ is the mechanism that copies data from one instance to another. The master site (publisher) will receive an update, it puts the update and a before and after image of the data in a queue. The slave sites (Subscribers) pull the data off the queue and apply them to the local database. The before and after images are used by replication to find the correct record and see if there are any update conflicts.

Besides Oracle replication, or your own home-grown replication, there are a lot of other uses for AQ.

You can drop a message on a queue for local usage. Say if you have a transactional system and are getting backed up but don't want to turn away incoming transactions. You can implement a queue. The receiving procedure can drop the transactions on the queue and a local de-queue procedure in the background can pull them off when it has time.

You can use AQ to interface with Java. AQ supports the Java Messaging Specification (JMS) API. Using Java, XML and AQ you can easily implement a SOA (service oriented architecture) web service.

What is Streams? Here's a brief description of streams and what you can use it for.

AQ and Replication both entail data movement. Streams is the current technology enabling that data movement. Streams is kind of like AQ, but with rules applied.

Let's think about AQ. AQ is basically a table and some table maintenance code wrapped around streams. When you enqueue a record, you're using AQ. Streams takes over and moves it to the next database, enqueueing it locally. AQ then takes over again, dequeuing it for consumption.

Streams has some nice features. I think the most important is the rule based transformations. A transformation allows you to modify the payload in flight. A receiving application doesn't need to be aware of the sending applications formats, it just receives what it needs.

Think of the way mainframes send data down to a data warehouse. The mainframe doesn't send entire vsam files down to let the warehouse figure out what pieces it needs. The warehouse group defines the fields it needs, a mainframe programmer writes a Cobol (maybe) program and sends a new, specific file down.

With streams, the receiving application can define what it needs and the sending application can define rules to match. The nice thing with streams is that there can be multiple consumers receiving the same payload but have different rules applied for them. The sender sends one payload and it's transformed many times in different ways for multiple consumers. Rules are also easily defined as opposed to writing a program, scheduling a batch processing window, writing a load routine, etc. With streams, identify the source, define the rules and write a consumer dequeue.

Speaking of data warehouses, another use of streams is in change data capture (CDC). You identify a source object, say your transaction detail table in the oltp system. You can create a rule that says capture all transactions that are approved and billable. Define the billing warehouse as a consumer for that stream. That payload can be applied to a staging table for loading into a warehouse table. The apply to the staging table can be done without coding. With almost 0 lines of code you can move the data you want, i.e. billable items, from your oltp system directly to your warehouse.

Streams is also non-stressful to the source database. Streams reads the redo-logs and gathers information from that as opposed to running queries or DML against the source database's tables.

Ok, get to know what is AQ and stream and wanna setup locally, check it out from YeeThian' post.

Web 2.0? Web 2.86 ... 3.1... and beyond

2007-07-22T19:54:00.002-07:00

The explosion of blogs, social networks and user-generated media commonly referred to as Web 2.0.

So what’s next? Web 3.0, or the notion of the semantic web, where the Internet evolves from a catalog of information to a smart network combining artificial intelligence, profiling and search capabilities.

O'Reilly editor Nat Torkington had been in Singapore last week, who were here giving the presentations on Web 2.0 and helping the IDA with their plans to foster startups in the country. As many people asked him about the future of Web 2.0--is it a bubble, when will it be replaced by something new, a recent blog post (The Future of Web 2.0) from him has answered that question in great detail.

In his post, future of web will go through the intersting changes till 2022 and experience the changes from web 2.0, web 286 (sounds like Pentium 286),web 3.0 & 3.1, Web 95, 98, NT, XP and Vista (Sounds like the revolution of the Microsoft Windows).

"The Future of Web 2.0" is an amusing blog post, whether or not you agree with his predictions about Firefox X, IE 13, WebVista, and other fancy terms, it is good to have some idea.

Enable the exception and stack trace in the OAS 10g 10.1.3

2007-07-22T19:54:00.001-07:00

In previous releases of OC4J, the default behavior when an error occurred in a Web application was to display both the exception and the stack trace in the HTML error page returned to the client.

This default behavior has changed in the oracle application server 10g 10.1.3 and these details are no longer displayed by default; instead, a generic error message is displayed in the HTML error page. The exception and stack trace details are sent to the log file of the relevant application.

If you have tests that rely on the display of an exception or stack trace, you can cause the stack trace to be displayed by running the application in developer mode. To run an application in development mode, set the development attribute of the <orion-web-app> element to "true" in the orion-web.xml file. Here is an example:
<?xml version="1.0"?>
<orion-web-app ... jsp-cache-directory="./persistence"
jsp-cache-tlds="standard"
temporary-directory="./temp"
context-root="/DevelopmentThrowException"
schema-major-version="10"
schema-minor-version="0"
development="true">
...
<web-app>
</web-app>
</orion-web-app>

To enable exception and stack track in oracle application server, you need to edit the orion-web.xml in the directory mentioned below once the web application is deployed into OAS 10g 10.1.3.

1.0 Find the orion-web.xml file from directory of <oracle_app_server_home>/j2ee/application-deployments/<instance_name>/<web_application__ear_file_name>/<web_application_war_file_name>/orion-web.xml

2.0 Modify the entry of orion-web-app like:

<orion-web-app
...
development="true"
>
...
</orion-web-app>

Set the development attribute of the <orion-web-app> element to "true".

3.0 Restart the app server and remember that whenever you undeploy and redeploy the same web application, you need to redo this.

Here, i would like to thanks thiam hwa, who has pointed me to the right place of the solution, for his help.

Additional resources:

Oracle Containers for J2EE – Frequently Asked Questions

Exception and Stack Trace No Longer Displayed in HTML Error Page

Desklets for Java

2007-07-22T19:53:00.002-07:00

A desklet is a desktop widget written in Java, according to Joshua Marinacci who, along with Robert Cooper, released the first preview of AB5k, their Swing-based desklet toolkit.

Desktop widgets have been popular on OS X for a while, and are becoming familiar fixtures of desktops running Windows Vista. Most desktop widgets to date have been written for a native platform. Widgets written in Java, or "desklets," by contrast, can run on any platform for which a Java 6 runtime is available. The term "desklet" was coined by Joshua Marinacci and Robert Cooper, who recently released a preview of their first desklet toolkit, AB5k.

AB5k is a container for running desklets, word for widgets. Desklets are little programs that run inside the container and do one thing very well. Some common desklets include clocks, calendars, news reports, weather, searchers (like Wikipedia) and useless but fun little playthings.

What is ebXML?

2007-07-22T19:53:00.001-07:00

Electronic Business using eXtensible Markup Language, the 'eb' in ebXML stands for electronic business," and you can pronounce the phrase as "electronic business XML," "e-biz XML," "e-business XML," or simply "ee-bee-ex-em-el". ebXML is a family of XML based standards sponsored by OASIS and UN/CEFACT whose mission is to provide an open, XML-based infrastructure that enables the global use of electronic business information in an interoperable, secure and consistent manner by all trading partners.

ebXML was published in 1999 as an initiative of the United Nations Centre for Trade Facilitation and Electronic Business (UN/CEFACT) and the Organization for the Advancement of Structured Information Standards (OASIS). The original project envisioned five layers of data specification, including XML standards for:

-Business processes,
-Collaboration protocol agreements,
-Core data components,
-Messaging, &
-Registries and repositories

High-level overview of ebXML interaction between two companies (source: www.ibm.com, author:David Mertz)

ebXML is not itself a standard, rather, it is a container for several key specification standards administered by UN/CEFACT and OASIS.

Key ebXML standards include ebXML Messaging Services, ebXML Registry, ebXML Business Process Specification Schema and ebXML Collaboration Protocol Profile and Agreement (CPP/CPA).

The International Organization for Standardization (ISO) has approved the following five ebXML specifications as the ISO 15000 standard, under the general title, Electronic business eXtensible markup language:

ISO 15000-1: ebXML Collaborative Partner Profile Agreement
ISO 15000-2: ebXML Messaging Service Specification
ISO 15000-3: ebXML Registry Information Model
ISO 15000-4: ebXML Registry Services Specification
ISO 15000-5: ebXML Core Components Technical Specification, Version 2.01

OASIS technical committees and UN/CEFACT retain the responsibility for maintaining and advancing the above specifications.

The freebXML.org initiative was established to promote development and adoption of ebXML-based open-source software.

I highly suggest you to go through the article called "Understanding ebXML, Untangling the business Web of the future", which can give you a very good idea of how each profile and specification related to each other.

Additional reading:

ebXMl Tutorial
http://www.javapassion.com/webservices/ebXML4.pdf

Integrating Java Content Repository and Spring

2007-07-22T19:50:00.000-07:00

Integrating Java Content Repository and Spring is a detailed infoQ article by Costin Leau which first talks about the Java Content Repository (JCR) spec in general, and then shows how the JCR support in Spring Modules provides a number of benefits when working against any repository supporting the JCR standard.

How to select Code Analysis Tools

2007-05-24T20:53:00.002-07:00

Code checking tools are software products that check compile-time whether a software program adheres to a certain set of rules of a coding standard. Without such code checkers, the enforcement of a coding standard in an organization is likely to fail. There are two main causes for this: (1) the number of rules in a coding standard is usually so large that nobody can remind them all (except the author) and (2) some context-sensitive rules that demand inspection of several files are very hard to check by human beings.

Although code checkers can reduce formal code review time considerably, it certainly does not mean that code reviews are not needed anymore. Some coding rules cannot be checked at all (e.g. "Comments should be in English") or only partly at compile-time (e.g. "Use delete, whenever you use new"). On average about 65% of a coding standard can be covered with code checking tools.

There are several characteristics that should be assessed during the selection process of a code checker. There are 6 issues that are most important. Obvious requirements such as whether it is available for the used platform are not taken into account.

Availability to define own rules. Experience has shown that only about 20% of a propietary coding standard is available as built-in rule in a code checker. As a consequence, there should be a possibility to define own rules. This can be via an API (complex in use but powerful) or a graphical user interface (easy to use but limited).

Integration in programming IDE. Most end users of code checkers are software engineers. Therefore a code checker should fit smoothly in the used programming environment. Preferrably via a toolbar with an easy way of running the tool (one button push).

Presence of command-line version. In order to be able to integrate the code checker in the software development process and/or in the programming IDE (if no plug-in is available) it should be command-line oriented. Command-line versions also allow for nightly batch runs of the code checker to collect quality data.

High performance. Code checkers are slower than compilers by nature because they perform a more in-depth semantical analysis of the source code. However, performance is a key issue for end user acceptance of the tooling. Response times of more than 1 minute per file are perceived as not workable.

Low cost of ownership. Cost of ownership means purchase price and maintenance costs. Since code checkers are not used continuously by end users, floating licenses appear to be more cost effective in most cases. About 1 floating end user license is needed per 10 developers.

Available support. Decent product support is an essential feature of a code checker. Good support does not only mean prompt and adequate response to user questions and problems, but also the frequency of new product release. "We will solve this in the next release" is not interesting if the average product release cycle is 1 year.

This How-to is an excerpt from a article called "Industrial methodology to introduce coding standards" and i feel the considerations and points are logical and good to follow for the code analyzers evaluation.

Evaluation of three open source code analysers

2007-05-24T20:53:00.001-07:00

There are couple of code analysis tools avaiable out there and three of them (FindBugs, PMD and QJPro) i have evaluated and would like to elaborate some more. I will give a quick demo for each along the way and it will give you a good visualization of how each tool looks like in the real life.

FindBugs

FindBugs looks for bugs in java the programs. It can detect a variety of common coding mistakes, including thread synchronization problems, misuse of API methods and etc.

FindBugs is a static analysis tool that examines java classes or JAR files and looks for potential problems by matching class bytecodes against a list of bug patterns. With static analysis tools, we can analyze software without actually running the program. FindBugs requires JRE (or JDK) 1.4.0 or later to run. However, it can analyze programs compiled for any version of Java. FindBugs supports a plugin architecture allowing anyone to add new bug detectors, which means we can write custom detectors to find application-specific problems and it has been used for Eclipse and Java JDK project for bugs finding.

We can run FindBugs in multiple ways -- from a GUI, from a command line, using Ant, as an Eclipse plug-in, and using Maven. However, it makes more sense to run it as an integrated part of nightly build infrastructure and generate a bug report, which could be easily done by using tools like CruiseControl, Anthill, or even simple cron to fire off nightly builds and giving the results as an email. FindBugs certainly won't find all the bugs, but they'll help find some of them. FindBugs can act as a safety net and detects identified bug patterns when developers check in their codes into the cvs. You can find a demo for FindBugs here and a viewlet here.

PMD

PMD is not like FindBugs and it analyzes the java source code instead of class files, scans and looks for potential problems like:

* Possible bugs - empty try/catch/finally/switch statements
* Dead code - unused local variables, parameters and private methods
* Suboptimal code - wasteful String/StringBuffer usage
* Overcomplicated expressions - unnecessary if statements, for loops that could be while loops
* Duplicate code - copied/pasted code means copied/pasted bugs

PMD has over 175 “rules”, which are undesirable patterns of code, right out of the box. These rules are grouped into “rulesets,” such as Basic, Unused Code Design, Optimization, Strict Exception, String and StringBuffer, Naming, JUnit and Java Logging.

Not all of them will be appropriate for our project or organization. Fortunately, PMD is configurable and we can configure which rules we want enforced. To do this, we can make our own ruleset file, which states which of the rules we want executed during analysis.

PMD is integrated with Eclipse, JBuilder, IntelliJ IDEA, Maven, Ant, and JDeveoper release 10.1.2 and 10.1.3.

It can also be run on demand from the command line, which allows it to be integrated into nightly Ant/Maven builds and publish the results to a shared location such as an intranet or file server.

There are two ways we can install the PMD plugin for JDeveloper release 10.1.3. One way is to install the extension by using JDeveloper's built in "update center" functionality. Fire up JDeveloper, click "Help", click "Check for Updates", then click "Add" to add a new update center and enter:

Name: PMD Update Center
Location: http://pmd.sf.net/center.xml

Or download the binary release here and unzip it into your JDEV_HOME\jdev\extensions.

To use the PMD, open the Tools-&Preferences menu, click on the PMD option, and select a couple of rules to try. To run it, right click on either a Java source file or a .jpr file (i.e., a project file) and select PMD. Any rule violations should show up in a LogWindow at the bottom of the screen. You can find the viewlet for the PMD here.

For FindBugs and PMD, these two tools focus on different aspects of software quality. Tools like PMD are extremely valuable on enforcing consistent coding style guidelines, and making code easier to understand by developers. Tools like FindBugs help uncover errors, while largely ignoring style issues. Therefore, PMD and FindBugs complement each other, and neither is a good substitute for the other.

There are some comments about FindBugs and PMD, which i found out from the www.theserverside.com and give a good explanation of both as well.

"FindBugs is a very good partner with PMD because it finds issues that are "deeper" than PMD's analysis. Its explanations of the bugs it finds are excellent. The most valuable bugs it picks up for my team are missing stream closures and inconsistent synchronization behavior. We removed the EI and EI2 rules, which spend a lot of time complaining about methods that return internal arrays, which in practice is not all that evil IMHO."

"PMD is definitely the most "complete" of all the tools mentioned here. Its customizability via XSL rules or code is great, the core rulesets are broad in scope and well thought-out, it works well, and has a good Eclipse plugin. Many of the rules are fairly shallow, insofar as they are more stylistic than genuine bug-analysis rules; you also will definitely want to pick and choose which rules to use. We found it pretty easy to select a ruleset between PMD and FindBugs that we all agreed in theory having zero violations was an attainable goal. We're currently just under one failure per 100 lines of code, which is the best of any commercial project I've been on."

QJPro

QJPro is very much similar to PMD and the reason i want to mention here is because QJPro prodvide a Plugin for JDeveloper 9.0.x which PMD did not have. If developers who are using a lower version of JDeveloper like release 9.0.x and want to have a plugin like PMD, maybe QJPro can be considered. You can find the viewlet for the QJPro here.

J2EE Application Performance Tuning Approach

2007-05-24T20:52:00.001-07:00

1.0 Set a goal: such as the response time for a given screen transaction per user. or transaction throughput, which is the number of transactions in a given time period, usually one second. For example, you could have a performance measurement that could be no more than three seconds for each screen form or a transaction throughput of one hundred transactions in one second. You should be able to find out this kind of information from project service-level or quality of service (QoS) requirements.

2.0 Identify the bottlenecks: For example, if the CPU usage on an application server is high, you will want to focus on tuning the application server first. There are some simple ways to identify the problem areas, check out my another post for some quick techniques.

3.0 Follow a methodical path: It is better spent tuning a method that takes 10 seconds but gets called 100 times than tuning a method that takes one minute but gets called only once. Make one change and stress-test it, if the change results in positive impact, only then will you make it permanent.

Two kinds of tools that are helpful in the tunning process, namely stress tools that generating load to your application and monitoring tools that collecting data of various performance indicators. For a comprehensive list of stress and monitoring tools, check out Software QA/Test Resource Center.

This is a good article about tunning strategy for the apache, tomcat & application itself, check it out. If none of the solution resolve the bottlenecks, you can consider scaling up the server (database & application). If horizontal scaling is not possible, consider the option of vertical scaling.

By the way, there is one tool I would like to mention here is the JMeter - a free tool from Apache community. It Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types. It can also make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load. To get to know it, check out this demo from my colleague - Srini.

J2EE Application Performance Tuning and Optimization

2007-05-24T20:48:00.000-07:00

When an application is deployed into the production environment, there are many factors contribute to the ultimate application performance & throughput, not only from application iteself, but types of hardward, settings and configurations of os and servers( web, application & database) and even the way of network setup also play a part.

As there are many places may degrade the performace, approches and solutions will be different and various. Maintaining a list of articles and papers about these approches and soutions will be very helpful & useful in our team and I would like to put some effort to maintain a such list of articles available in the internet regarding the application side(E.g programming practise, application server configurations, technology usage and etc).

Performance tuning is as much as it is a science, team work will be more productive and enjoyable than one man show. If you are the people like me and would like to put some effort into this, let me know and maybe in future we can co-author a paper or a course ( English/Chinese) based on our experience and knowledge gained. :)

In the meantime, i would like to share some of my own experience in the performance tuning and optimization.

In the reality, when expecting application performance & throughput is not met, no party in the project would like to admit the issues or problems are from them since people don't like changes and project deadline is around the corner. In the end, the deadline is extened and peoples from different team (application, infrastructure, database) have to sit together to look into the case.

No matter which team are responsible for it, the most important thing is to find out where & what the bottlenecks are degrading the performance. Bare in mind that there maybe more than one bottleneck exist and no point to complain other people. There are some verifications we can do to find out them one by one until the output meets the expectation.

To verify whethere the network is the cause, we can try to ping or ftp an big file to the target machine to get to know the network speed. If the speed is very slow, it may be one of bottleneck degrading the application, we can inform the infrastrucre team to review their network routing strategy.

To verfiy whethere servers are running or responding slowly, there are a few things we need to look out. We need to know the hardward type, os configuration and server installation version. Especially in the staging environment, one machine may be shared by multiple teams and applcaitons and people can cap the cpu & memeory resource for certain process such as the database server, application server and even your deployed applicaton without knowing it. People can tell you that the machine running the application or servers is a very fast and even with dual processor, but it doesnot mean you are having full power of that machine.

Other than the OS configurations, you also need to know the installation version of server (database, application) and whether they has been installed and setup with recommened parameters and models.

For the application, usually the performace issues are from inefficient business logic, bad sql statement (E.g not using prepared statement & execute batch and commit too frequently, etc.) and db connection handling (connection pooling, leaking, jdbc driver). Usually, a code review will be very helpful to spot those problems. For the sql statement, it is best to consult with DBA to tune the sql (such as truncating a table VS deleting all records). To spot whether there is a db connection leaking, run the "netstat" in the command line and verfiy if there are any outstanding db connection ports opening after db works are done. If the problem still can not be identified, you can use tools like application profiler to spot the bottleneck from certain method or function of the application and narrow down until you find the problem.

What is ebXML?

2007-05-24T20:46:00.000-07:00

-Business processes,
-Collaboration protocol agreements,
-Core data components,
-Messaging, &
-Registries and repositories

ebXML is not itself a standard, rather, it is a container for several key specification standards administered by UN/CEFACT and OASIS.

Key ebXML standards include ebXML Messaging Services, ebXML Registry, ebXML Business Process Specification Schema and ebXML Collaboration Protocol Profile and Agreement (CPP/CPA).

OASIS technical committees and UN/CEFACT retain the responsibility for maintaining and advancing the above specifications.

The freebXML.org initiative was established to promote development and adoption of ebXML-based open-source software.

Additional resources:

Understanding ebXML, Untangling the business Web of the future.
http://www-128.ibm.com/developerworks/xml/library/x-ebxml/
ebXMl Tutorial
http://www.javapassion.com/webservices/ebXML4.pdf

Java/XML rich-client technology

2007-05-24T20:07:00.000-07:00

Rich client technology/platform like visual basic and java swing, developers are very much familiar and other technologies like ajax (Google Web Tookit & ECHO2), Eclipse RCP and XUL (Mozila) are also well-known for years.

Recently, I came across an article talking about java/xml rich client technology and its online demonstration impressed me very much in its richness of client-side widgets and functionalities. It runs inside a browser, but pretty muck like a appliaction running on the desktop.

Not like XUL, it does not need a XUL motor (such as Mozila) to provide a running platform, but using jvm plugin in the browser, which allows it to be used in different browsers with jvm plugin.

Currently,two companies are providing this kind of technology, which are nexaweb and asperon. Their online demos are worth for a look.

Enterprise Console Demo From Nexaweb
http://demo.nexaweb.com/Nexaweb/start.jsp?app=sanmgr/index.xml&nocache=true

Order Management Demo From Nexaweb
http://demo.nexaweb.com/Nexaweb/nopushstart.jsp?app=orderdemo/orderStatus.do&nocache=true

CRM Demo from Asperon
http://www.asperon.net/crm/index.html

There are some other interesting rich client demos which is using XUL or Ajax. Check it out.

Ajax Demo from Echo2
http://demo.nextapp.com/Demo/app

XUL Demoonstration
http://xul.sourceforge.net/challenge.html

Reference:
Wei C. (2004). XML Rich Client Technology Strengthens Java. Last retrieved from
http://www.computerworld.com/developmenttopics/development/story/0,10801,93858,00.html?SKC=

Wei C. (2004). Java or .NET? XML Rich-Client AJAX Technology Brings Zero-Install Rich Client To Java
Last retrieved from http://jdj.sys-con.com/read/111208_2.htm

Countries with My Footprints

2007-05-24T19:46:00.000-07:00