Garbage Collector guidelines and tips

These are some of the guidelines and tips I usually look at when I need to tune the GC.

Are mostly taken by the following 2 books and few of them from my experience:

• Java Performance
• JBoss AS 5 Performance Tuning

Hopefully they will be useful for someone else out there :)

Garbage Collector

XX:+AggressiveOpts sets a HotSpot internal boolean variable to true to enable additional performance optimizations

***************************************************************************************************

Since most objects are locked by at most one thread during their lifetime, enabling -XX:+UseBiasedLocking allows that thread to bias the lock toward itself. Once biased, that thread can subsequently lock and unlock the object without resorting to expensive atomic instructions

***************************************************************************************************

To print the default ergonomic values:

java -XX:+PrintCommandLineFlags -version

***************************************************************************************************

-XX:+PrintGCDetails prints additional and more valuable garbage collection information.

An example of -XX:+PrintGCDetails output from Java 6 Update 25’s throughput garbage collector, enabled via -XX:+UseParallelGC or -XX:+UseParallelOldGC, is shown in the following example. The output is spread across several lines for easier reading.

[GC
   [PSYoungGen: 99952K->14688K(109312K)]
   422212K->341136K(764672K), 0.0631991 secs]
   [Times: user=0.83 sys=0.00, real=0.06 secs]

***************************************************************************************************

Including Date and Time Stamps

-XX:+PrintGCTimeStamps

YYYY-MM-DD-T-HH-MM-SS.mmm-TZ

When -XX:+PrintGCDetails is used in combination with -Xloggc:<filename>, the output is automatically prefixed with a time stamp even without specifying -XX:+PrintGCTimeStamps.

***************************************************************************************************

[Full GC (System)
   [PSYoungGen: 99608K->0K(114688K)]
   [PSOldGen: 317110K->191711K(655360K)]
   416718K->191711K(770048K)
   [PSPermGen: 15639K->15639K(22528K)],
   0.0279619 secs]
   [Times: user=0.02 sys=0.00, real=0.02 secs]

System means there is a System.gc() in the code

***************************************************************************************************

http://sysadminsjourney.com/2008/09/15/profile-gc-with-gchisto/

***************************************************************************************************

VisualVM and VisualGC plugin

if remote u need to install jstatd daemon

**************************************************************************************************

The tiered server runtime is enabled with the -server -XX:+TieredCompilationcommand line options.

Tip

If you do not know which runtime to initially choose, start with the server runtime. If startup time or memory footprint requirements cannot be met and you are using Java 6 Update 25 or later, try the tiered server runtime. If you are not running Java 6 Update 25 or later, or the tiered server runtime is unable to meet your startup time or memory footprint requirement, switch to the client runtime.

**************************************************************************************************

32-Bit or 64-Bit JVM

Operating System	Java Heap Size	32-Bit or 64-Bit JVM
Windows	Less than 1300 megabytes	32-bit
Windows	Between 1500 megabytes and 32 gigabytes[*]	64-bit with -d64 -XX:+UseCompressedOopscommand line options
Windows	More than 32 gigabytes	64-bit with -d64command line option
Linux	Less than 2 gigabytes	32-bit
Linux	Between 2 and 32 gigabytes[*]	64-bit with -d64 -XX:+UseCompressedOopscommand line options
Linux	More than 32 gigabytes	64-bit with -d64command line option
Oracle Solaris	Less than 3 gigabytes	32-bit
Oracle Solaris	Between 3 and 32 gigabytes[*]	64-bit with -d64 -XX:+UseCompressedOopscommand line options
Oracle Solaris	More than 32 gigabytes	64 bit with -d64command line option

[*] Best performance in the 64-bit HotSpot VM with -XX:+UseCompressedOops is realized around 26 gigabytes or less of maximum Java heap size. HotSpot VM versions later than Java 6 Update 18 automatically enable -XX:+UseCompressedOops by default based on maximum Java heap size.

**************************************************************************************************

start with the parallelOldDC : young and old  collection  is multithreading

The throughput garbage collector is specified by the HotSpot VM command line option -XX:+UseParallelOldGC or -XX:+UseParallelGC. If -XX:+UseParallelOldGC is not available in the version of the HotSpot VM you are using, use -XX:+UseParallelGC. The difference between the two is that -XX:+UseParallelOldGC enables both a multithreaded young generation garbage collector and a multithreaded old generation garbage collector, that is, both minor garbage collections and full garbage collections are multithreaded. -XX:+UseParallelGC enables only a multithreaded young generation garbage collector. The old generation garbage collector used with -XX:+UseParallelGC is single threaded. Using -XX:+UseParallelOldGC also automatically enables -XX:+UseParallelGC. Hence, if you want to use both a multithreaded young generation garbage collector and a multithreaded old generation garbage collector, you need only specify -XX:+UseParallelOldGC.

**************************************************************************************************

garbage collection logging:

-XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:<filename>

When tuning the HotSpot VM for low latency the following two command line options are useful since they report the amount of time the application has been blocked due to a VM safepoint operation and how long the application has executed between safepoint operations.

• -XX:+PrintGCApplicationStoppedTime
• -XX:+PrintGCApplicationConcurrentTime

option -XX:+PrintSafepointStatistics can help distinguish garbage collection safepoints from other safepoints.

The -XX:+PrintGCApplicationConcurrentTime command line option can be used to determine whether the application was executing, and for how long, during some time period of interest where an observed response time exceeds application requirements.

**************************************************************************************************

Recommended GC Logging Command Line Options

GC Command Line Option	Most Applicable
-XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:<filename>	Minimal set of command line options to enable for all applications.
-XX:PrintGCDateStamps	Use when wanting to see a calendar date and time of day rather than a time stamp indicating the number of seconds since the JVM was launched. Requires Java 6 Update 4 or later.
-XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+PrintSafepointStatistics	Useful when tuning an application for low response time/latency to help distinguish between pause events arising from VM safepoint operations and other sources.

XX:+PrintCommandLineFlags prints the selected initial and maximum heap sizes at HotSpot VM initialization time as -XX:InitialHeapSize=<n>

-XX:+PrintTenuringDistribution

****************************************************************************************************************************

-XX:MaxHeapSize=<m>, where <n> is the initial Java heap size in bytes and <m> is the maximum Java heap size in bytes

****************************************************************************************************************************

Space	Command Line Option	Occupancy Factor
Java heap	-Xms and -Xmx	3x to 4x old generation space occupancy after full garbage collection
Permanent Generation	-XX:PermSize -XX:MaxPermSize	1.2x to 1.5x permanent generation space occupancy after full garbage collection
Young Generation	-Xmn	1x to 1.5x old generation space occupancy after full garbage collection
Old Generation	Implied from overall Java heap size minus the young generation size	2x to 3x old generation space occupancy after full garbage collection

****************************************************************************************************************************

Young space

 

-XX:NewSize=<n>[g|m|k]

-XX:MaxNewSize=<n>[g|m|k]

-Xmn<n>[g|m|k]

-Xmn is convenient to size both the initial and maximum size of the young generation space

if  (-Xmx != -Xms) && -Xmn is present then

a growth or contraction in the Java heap size will not adjust the size of the young generation space.

The size of the young generation space will remain constant with any growth or contraction of the Java heap size. Therefore, -Xmn should be used only when -Xms and -Xmx are set to the same value.

****************************************************************************************************************************

perm size

-XX:PermSize=<n>[g|m|k]

-XX:MaxPermSize=<n>[g|m|k]

Java applications with an emphasis on performance should size both the initial and maximum permanent generation sizes (-XX:PermSize and -XX:MaxPermSize) to the same value since growing or contracting the permanent generation space requires a full garbage collection.

****************************************************************************************************************************

-XX:-ScavengeBeforeFullGC will disable young generation space garbage collection on full garbage collections.

****************************************************************************************************************************

2010-11-25T18:51:03.895-0600: [Full GC
   [PSYoungGen: 279700K->267300K(358400K)]
   [ParOldGen: 685165K->685165K(685170K)]    964865K->964865K(1043570K)
   [PSPermGen: 32390K->32390K(65536K)],
   0.2499342 secs]
   [Times: user=0.08 sys=0.00, real=0.05 secs]Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

2010-11-25T18:26:37.755-0600: [Full GC
   [PSYoungGen: 0K->0K(141632K)]
   [ParOldGen: 132538K->132538K(350208K)]
   32538K->32538K(491840K)
   [PSPermGen: 65536K->65536K(65536K)],    0.2430136 secs]
   [Times: user=0.37 sys=0.00, real=0.24 secs]java.lang.OutOfMemoryError: PermGen space

If you observe an OutOfMemoryError in the garbage collection logs, try increasing the Java heap size to 80% to 90% of the physical memory you have available for the JVM.

For example, increase -Xmsand -Xmx for old generation space OutOfMemoryErrors, and increase -XX:PermSize and -XX:MaxPermSize for permanent generation OutOfMemoryErrors.

the next step is to calculate the application’s live data size.

****************************************************************************************************************************

Calculate Live Data Size

In addition to the live data size, the full garbage collections at steady state also provide the worst case latency to expect due to full garbage collections.

you use the live data (perm space + old space after full GC during application busy time) to calculate the initial heap size :

As a general rule, the initial and maximum Java heap size command line options, -Xms and -Xmx, should be set to a value between three and four times larger than the live data size of the old generation space.

old generation space occupancy after the full garbage collection is 295111K, or about 295 megabytes Hence, the live data size is about 295 megabytes

Therefore, the suggested initial and maximum Java heap size to specify for this application should be a value between 885 and 1180 megabytes, that is, -Xms1180m -Xmx1180m for four times the live data size.

Also as a general rule, the initial and maximum permanent generation size, -XX:PermSize and -XX:MaxPermSize, should be 1.2x to 1.5x larger than the live data size of the permanent generation space.

In the example full garbage collection shown in the picture above, the permanent generation space occupancy after the full garbage collection is 32390K, or about 32 megabytes. Hence, the suggested initial and maximum permanent generation space size to specify for this application should be between 38 megabytes and 48 megabytes, that is, -XX:PermSize=48m -XX:MaxPermSize=48m, for 1.5 times the permanent generation live data size.

As an additional general rule, the young generation space should be 1 to 1.5 times the old generation space live data size.

As a result, the suggested young generation size should be between 295 and 442 megabytes. In the picture above, the young generation space size is 358400K, about 358 megabytes. 358 megabytes is within the recommended size.

If the initial and maximum Java heap size is 3x to 4x the live data size and the young generation space is 1x to 1.5x the live data size, the size of the old generation space should be between 2x to 3x the live data size.

The combined Java command line applying these general sizing rules based on the garbage collection data in  the picture above is

$  java -Xms1180m -Xmx1180m -Xmn295m
       -XX:PermSize=48m -XX:MaxPermSize=48m

Space	Command Line Option	Occupancy Factor
Java heap	-Xms and -Xmx	3x to 4x old generation space occupancy after full garbage collection
Permanent Generation	-XX:PermSize -XX:MaxPermSize	1.2x to 1.5x permanent generation space occupancy after full garbage collection
Young Generation	-Xmn	1x to 1.5x old generation space occupancy after full garbage collection
Old Generation	Implied from overall Java heap size minus the young generation size	2x to 3x old generation space occupancy after full garbage collection

****************************************************************************************************************************

Tune Latency/Responsiveness

The following activities are involved in evaluating the garbage collector’s impact on latency:

• Measuring minor garbage collection duration
• Measuring minor garbage collection frequency
• Measuring worst case full garbage collection duration
• Measuring worst case full garbage collection frequency

Refine Young Generation Size

from parallelGC to concurrent  -XX:+UseConcMarkSweepGC  

if the worst case full garbage collection duration or frequency  collection frequencies are too high

Additional general guidelines to keep in mind as the young generation size is changed are

• The old generation space size should be not be much smaller than 1.5x the live data size.See the previous section “Determine Memory Footprint” for live data size definition and additional old generation sizing guidelines.
• Young generation space size should be at least 10% of the Java heap size, the value specified as -Xmx and -Xms. A very small young generation size can be counterproductive. It leads to frequent minor garbage collections.
• When increasing the Java heap size, be careful not to exceed the amount of physical memory available to the JVM. A Java heap size that consumes enough memory to cause the underlying system to swap to virtual memory results in poor garbage collector and application performance.

Refine Old Generation Size

The objective of this task is to evaluate the worst case pause time induced by a full garbage collection and the frequency of full garbage collections.

If You Are Observing Only Full Garbage Collections

When modifying the size of the old generation space, it is possible the old generation size may become out of balance with the young generation size and result in the application experiencing only full garbage collections. Usually this occurs when the old generation space is not large enough to hold all the objects being promoted from the young generation space, even after a full garbage collection.

The key indicator that the old generation space is not large enough is that little space has been reclaimed in the old generation space (the values to the right of the ParOldGen label), and a large portion of the young generation space remains occupied after each full garbage collection. When not enough space is available in the old generation to handle promoted objects from the young generation, objects “back up” into the young generation space as observed in the preceding output.

If you are not able to meet your application’s worst case latency requirements due to full garbage collection duration being too long, then you should switch to using the concurrent garbage collector.

The concurrent garbage collector is enabled with the HotSpot command line option:

-XX:+UseConcMarkSweepGC

****************************************************************************************************************************

Avoiding survivor space overflow is accomplished by sizing the survivor spaces so they are large enough to hold surviving objects long enough to age for some period of time. Effective aging results in only long-lived objects being promoted to the old generation space.

Tip

Aging is the means by which objects are retained in the young generation until they are no longer reachable, so as to preserve old generation space for longer-lived objects.

The survivor spaces are sized using the HotSpot command line option:

-XX:SurvivorRatio=<ratio>

The <ratio> must be a value greater than 0. -XX:SurvivorRatio=<ratio> expresses the ratio of space between each survivor space and the eden space. The following equation can be used to determine the survivor space size:

survivor space size = -Xmn<value>/(-XX:SurvivorRatio=<ratio> + 2)

The larger the value specified as the ratio, the smaller the survivor space size.

Tenuring Threshold Explained

Tip

Effective object aging in the young generation to prevent them from being prematurely promoted to the old generation space reduces the rate that the old generation occupancy increases. This reduces the frequency at which the CMS garbage collection cycle must execute and also reduces the likelihood of fragmentation.

There is also a HotSpot VM command line option, -XX:MaxTenuringThreshold=<n>, that can be used to ask the HotSpot VM to promote objects to the old generation space only after an object’s age exceeds the value of <n>.

The max tenuring threshold can be set to a value ranging from 0–15 for Java 5 Update 6 and later, 0–31 for Java 5 Update 5 and earlier.

It is not recommended to set the max tenuring threshold value to 0. This causes objects to be immediately promoted from young generation to old generation on the next minor garbage collection after an object has been allocated. This will grow the old generation space very rapidly and result in frequent full garbage collections.

It is also not recommended to set the max tenuring threshold to a value larger than the possible maximum. That will result in objects being retained in survivor spaces until survivor spaces overflow. If they overflow, objects are promoted to the old generation nondiscriminantly, that is, they are not promoted based on their age. As a result, short-lived objects may be promoted before longer-lived objects, which prevents effective object aging.

Tip

In general, observing a new tenuring threshold value that is consistently less than the max tenuring threshold or observing a desired survivor size that is smaller than the number of total surviving bytes (the value for the last row of object ages and the far right column) are indications that the survivor spaces are too small.

****************************************************************************************************************************

Initiating the CMS Collection Cycle

Stop-the-world compacting garbage collections are the worst case garbage collection induced latency

The initiation of a CMS cycle is based on the occupancy of old generation space

If you observe stop-the-world compacting garbage collections, you can tune when the CMS cycle should start. Stop-the-world compacting garbage collections in CMS are identified in garbage collection output by concurrent mode failure. The following is an example:

174.445: [GC 174.446: [ParNew: 66408K->66408K(66416K), 0.0000618
secs]174.446: [CMS (concurrent mode failure): 161928K->162118K(175104K),
4.0975124 secs] 228336K->162118K(241520K)

f you are observing concurrent mode failures in your garbage collection output, you can instruct the HotSpot VM to initiate the start of the CMS cycle earlier using the command line option:

-XX:CMSInitiatingOccupancyFraction=<percent>

The value specified is the percentage of old generation occupancy at which the CMS garbage collection cycle should start. For instance, if you would like the CMS cycle to start at an old generation space occupancy of 65%, you set -XX:CMSInitiatingOccupancyFraction=65. A second HotSpot command line option should be used in conjunction with -XX:CMSInitiatingOccupancyFraction=<percent> called

-XX:+UseCMSInitiatingOccupancyOnly

-Xmx1536m -Xms1536m -Xmn512m
-XX:CMSInitiatingOccupancyFraction=51
-XX:+UseCMSInitiatingOccupancyOnly

Explicit Garbage Collections

If you observe full garbage collections, which are initiated by an explicit call to System.gc(), there are two ways to deal with them when using the concurrent garbage collector:

1. You can request the HotSpot VM to execute them as a concurrent garbage collection cycle using the HotSpot VM command line option:
2. -XX:+ExplicitGCInvokesConcurrent
3. or
4. -XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses
5. The first requires Java 6 or later. The second requires Java 6 Update 4 or later. It is generally better to use -XX:+ExplicitGCInvokesConcurrentAndUnloadsClasses if the JDK version you are using supports it.
6. You can ask the HotSpot VM to ignore explicit calls to System.gc() by using the Hotspot command line option:
7. -XX:+DisableExplicitGC
8. This command line option also ignores explicit calls to System.gc() in the other HotSpot VM garbage collectors.

The HotSpot VM by default does not garbage collect the permanent generation space with CMS despite the CMS Perm label reported in the garbage collection output. To enable CMS permanent generation garbage collection, you must specify the following HotSpot VM command line option:

-XX:+CMSClassUnloadingEnabled

If you are using Java 6 Update 3 or earlier, you must also specify the following command line option in addition to -XX:+CMSClassUnloadingEnabled:

-XX:+CMSPermGenSweepingEnabled

CMS Pause Time Tuning

The number of threads used in the remark phase can be controlled by the following HotSpot VM command line option:

-XX:ParallelGCThreads=<n>

The duration of the remark phase can in some cases be reduced by specifying

-XX:+CMSScavengeBeforeRemark

This command line option forces the HotSpot VM to perform a minor garbage collection prior to a CMS remark. Doing a minor garbage collection just prior to a remark can minimize the amount of work for the remark phase by reducing the number of objects in the young generation space that may be reachable from the old generation space.

If the application has a large number of Reference or finalizable objects to be processed, specifying the following HotSpot VM command line option can help reduce garbage collection duration:

-XX:+ParallelRefProcEnabled

Latest and Greatest Optimizations

When new performance optimizations are integrated into the HotSpot VM they are usually introduced under the command line option -XX:+AggressiveOpts.

Using the -XX:+AggressiveOpts command line option should be considered if the application stakeholders are looking for additional performance and are willing to accept the additional small risk associated with enabling the most recent optimizations.

Escape Analysis

Escape analysis is a technique that evaluates the scope of a Java object. In particular, if a Java object allocated by some executing thread can ever be seen by a different thread, the object “escapes.” If a Java object does not escape, additional optimization techniques can be applied. Hence, the optimization technique is called escape analysis.

Escape analysis optimizations in the HotSpot VM are enabled with the following command line option:

-XX:+DoEscapeAnalysis

Large Pages on Linux

-XX:+UseLargePages