Java Heap, Stack Size, Perm Gen and Full GC tunning

A good start to understand JVM, Heap, Perm Gen and GC collector

JDK 6 Performance Features and Update

GC Collector:

Serial Collector (-XX:+UseSerialGC)

• Throughput Collectors

> Parallel Scavanging Collector for Young Gen

 -XX:+UseParallelGC

> Parallel Compacting Collector for Old Gen

 -XX:+UseParallelOldGC (on by default with ParallelGC in JDK 6)

• Concurrent Collector

> Concurrent Mark-Sweep (CMS) Collector

 -XX:+UseConcMarkSweepGC

> Concurrent (Old Gen) and Parallel (Young Gen) Collectors

 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC

• The new G1 Collector as of Java SE 6 Update 14 (-XX:+UseG1GC)

Sample JVM Parameters:

Performance Goals and Exhibits

A) High Throughput (e.g. batch jobs, long transactions)

B) Low Pause and High Throughput (e.g. portal app)

• JDK 6

A) -server -Xms2048m -Xmx2048m -Xmn1024m -XX:+AggressiveOpts

-XX:+UseParallelGC -XX:ParallelGCThreads=16

B) -server -Xms2048m -Xmx2048m -Xmn1024m -XX:+AggressiveOpts

-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=16

• JDK 5

A) -server -Xms2048m -Xmx2048m -Xmn1024m -XX:+AggressiveOpts

-XX:+UseParallelGC -XX:ParallelGCThreads=16 -XX:+UseParallelOldGC

-XX:+UseBiasedLocking

B) -server -Xms2048m -Xmx2048m -Xmn1024m -XX:+AggressiveOpts

-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:ParallelGCThreads=16

-XX:+UseBiasedLocking

Rule of Thumb on Best selection of

> Garbage Collector (Make sure to override GCThreads)
> Heap Size (-Xms == 1/64 Max Memory or Max Heap and
-Xmx == ¼ Max Memory or Max Heap)
> Runtime Compiler (-server vs -client)
• Desired Goals (This is a hint, not a guarantee)
> Maximum Pause Time (-XX:MaxGCPauseMillis=)
> Application Throughput (-XX:GCTimeRatio= where
Application time = 1 / (1 + n))

Do check out 1.5 paper as it comes with sample parameters for high thoughput and low latency application

Java 1.5 Tuning White Paper

If you are using 1.6, check out the difference and improvement from 1.6 paper, take note that 1.5 parameters still apply to 1.6.

Java SE 6 Performance White Paper

A very practical and easy to understandable slide which walking you through the Sun HotSpot GC tuning tip and take-away parameters.

HotSpot JVM Tuning

Wondering whether java thread stack space (Xss) and  perm gen (MaxPermSize) part of heap space. The answer is NO. That is why when u saw actually linux memory consumption is bigger than your "-Xmx" settings.

Using -Xss to adjust Java default thread stack size to save memory and prevent StackOverflowError

One working jvm parameters with JBoss server:

JAVA_OPTS="-server -Xms3072m -Xmx3072m -Xmn2048m -XX:MaxPermSize=256m -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -XX:ParallelGCThreads=8 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31 -XX:+AggressiveOpts -XX:+PrintHeapAtGC -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -XX:+PrintGCApplicationStoppedTime -Xloggc:/opt/jbos/server/default/log/jvmgc.log"

Diagnosing a Garbage Collection problem

You have short-term memory on Linux Shell? Never mind, just check out this article and refresh your mind. 

Load Testing

Load Testing Calculator

Peak Hourly Visits and Average Visit Length

Peak Hourly Pages, Testcase Size and Duration

How Many Rows of Data Do I Need?

Case Studies 

Sample Reports

Load Testing Terms

Recommend Readings

How to do a capacity planning of web application and decide the deployment architecture

Oracle Archive Logging

[Original Post] Basically, any change that happens in the database is first captured in a memory structure called the Log Buffer. This memory structure exists inside the Oracle Instance. The Log Buffer normally has a small footprint (somewhere in the neighborhood of 1MB). Information in the Log Buffer memory is flushed to the Redo Logs by the background process LGWR under these curcumstances:
a) every 3 seconds
b) on a commit
c) when the Log Buffer becomes 1/3 full
d) on a checkpoint

Anytime a) b) c) or d) occurs in the database, the information from the Log Buffer is written to the current Red Log by LGWR. Redo Logs are actual physical files residing on the OS. When a Redo Log becomes full, a Log File Switch occurs and a pointer is set to start writing Log Buffer information to the next Redo Log in line. You can run your database with only 2 Redo Logs, but Oracle (and common sense) recommends at least 3. 
So, after this Log File Switch, you have a Redo Log that is full and a pointer to the next Relo Log. The Redo Log that is full needs to be archived (saved somewhere else). So now, a background process called ARCH will get a nudge from LGWR saying "hey, got a redo log that needs to be archived" and ARCH will pick it up and convert it to an Archived Log file and save it in the location specified by your init.ora parameter setting called log_archive_dest_1 (or in yor flash_recovery_area, depending).
So now that the previous Redo Log has been archived, it can be overwritten by LGWR when necessary (e.g., Redo Logs are written to in a round-robin fashion and when Redo Log #3 fills up, the pointer goes back around to Redo Log #1. So if you're in archivelog mode and redo log #1 hasn't been completely archived by ARCH yet, and LGWR needs to write to Redo Log #1, then your database "hangs" until that Redo Log #1 is freed up to be written to again).

So, what is the advantage of having Archived Logs? Say for example you expereince severe corruption or a database crash that required you to restore some datafiles from 7 hours ago. If you have all the archived logs from that pint in time (7 hours ago) up until the oment of the crash, you can apply (or "roll forward") all the changes contained in those archived logs against the restored datafiles. Basically this replays all the changes in the database over the past 7 hours. After recovering the last archived log, Oracle will then look to roll forward even more by using the online redo logs. If those online redo logs contain changes necessary, Oracle will apply those changes also. Basically, you can recover from a serious error all the way up to just before the error occurred. Minimal data loss is the advantage here. you can't do this when you're not in archivelog mode, because all the changes over the past 7 hours are lost because the redo logs just keep overwriting themselves and all the changes are lost between the time of your last backup and the time of the crash.
As the mantra goes . . . if you don't care if your database loses data, then run in noarchivelog mode. If you care about your data and don't want to lose it, then run the database in archivelog mode.

Memory, Swap, Process, Thread, File, Data Storage and Tunning

Physical and virtual memory

Traditionally, one has physical memory, that is, memory that is actually present in the machine, and virtual memory, that is, address space. Usually the virtual memory is much larger than the physical memory, and some hardware or software mechanism makes sure that a program can transparently use this much larger virtual space while in fact only the physical memory is available.

Nowadays things are reversed: on a Pentium II one can have 64 GB physical memory, while addresses have 32 bits, so that the virtual memory has a size of 4 GB. We'll have to wait for a 64-bit architecture to get large amounts of virtual memory again. The present situation on a Pentium with more than 4 GB is that using the PAE (Physical Address Extension) it is possible to place the addressable 4 GB anywhere in the available memory, but it is impossible to have access to more than 4 GB at once.


Swap Space


Linux divides its physical RAM (random access memory) into chucks of memory called pages. Swapping is the process whereby a page of memory is copied to the preconfigured space on the hard disk, called swap space, to free up that page of memory. The combined sizes of the physical memory and the swap space is the amount of virtual memory available.


Linux has two forms of swap space: the swap partition and the swap file. The swap partition is an independent section of the hard disk used solely for swapping; no other files can reside there. The swap file is a special file in the filesystem that resides amongst your system and data files

How big should my swap space be?

Linux and other Unix-like operating systems use the term "swap" to describe both the act of moving memory pages between RAM and disk, and the region of a disk the pages are stored on. It is common to use a whole partition of a hard disk for swapping. However, with the 2.6 Linux kernel, swap files are just as fast as swap partitions. Now, many admins (both Windows and Linux/UNIX) follow an old rule of thumb that your swap partition should be twice the size of your main system RAM. Let us say I've 32GB RAM, should I set swap space to 64 GB? Is 64 GB of swap space really required? How big should your Linux / UNIX swap space be?

RAID 0, RAID 1, RAID 5, RAID 10

RAID 5 vs RAID 10: Recommended RAID For Safety and Performance

System Tuning Info for Linux Servers

Disk Tuning
File system Tuning
SCSI Tuning
Disk I/O Elevators
Network Interface Tuning
TCP Tuning
File limits
Process limits
Threads
NFS
Apache and other web servers
Samba
Openldap tuning
Sys V shm
Ptys and ttys
Benchmarks
System Monitoring

limit a Linux process run on a particular core

taskset  -p 

i.e.

taskset 1 -p 12345

to set process 12345 to use only processor/core 1

The bitmask can be a list (i.e. 1,3,4 to use cores 1 3 and 4 of a 4+ core system) or a bitmask in hex (0x0000000D the 1,3,4, 0x00000001 for just core 1)

taskset is usually in a package called shedutils.

Edit: almost forgot... If you want to set the affinity of a new command instead of change it for an existing process, use:

taskset   []...[]

Maximum number of threads per process in Linux

number of threads = total virtual memory / (stack size*1024*1024)

Java Process & Thread Limit on Linux

Maximum threads managed by java

Heap size                                                                                            

Java Heap size does not determine the amount of memory your process uses

If you monitor your java process with an OS tool like top or taskmanager, you may see the amount of memory you use exceed the amount you have specified for -Xmx. -Xmx limits the java heap size, java will allocate memory for other things, including a stack for each thread. It is not unusual for the total memory consumption of the VM to exceed the value of -Xmx.

Garbage collection

There are essentially two GC threads running. One is a very lightweight thread which does "little" collections primarily on the Eden (a.k.a. Young) generation of the heap. The other is the Full GC thread which traverses the entire heap when there is not enough memory left to allocate space for objects which get promoted from the Eden to the older generation(s).

If there is a memory leak or inadequate heap allocated, eventually the older generation will start to run out of room causing the Full GC thread to run (nearly) continuously.

The amount allocated for the Eden generation is the value specified with -Xmn. The amount allocated for the older generation is the value of -Xmx minus the -Xmn. Generally, you don't want the Eden to be too big or it will take too long for the GC to look through it for space that can be reclaimed.