Intro
This page covers useful java CLI tools which can be used on any java process. Commands listed all connect to the java process. this will be in the form of command <PID> <options>. For further information man pages exist for these tools and can be read with man <java command>
common commands & uses.
jps -prints list of running java processes with process ID’s for each. Can also use musicstation-pid. This would be your first command to use getting the PID.
jstack -Prints Java stack traces of Java threads. Sometimes attempting to use JMX console to get thread dumps is unresponsive and therefore not always possible. jstack is an alternative if this occurs. you want to use this when we have alerts for high thread usage and high CPU usage.
To use run jstack -m <java PID>. You may want to redirect the output to a file as this will print out a lot of info to the terminal
jinfo - displays java configuration info for selected java process.
has a few options e.g. jinfo -flags <java PID>will display the JVM flags specified for the process. Which will appear as follows;
| -+jinfo -flags +--+27592+- | |||||||||
| -+Attaching to process ID + | |||||||||
| -+Debugger attached successfully.+- | |||||||||
| -+Server compiler detected.+- | |||||||||
| -+JVM version is + | |||||||||
| -+-Dprogram.name=run.sh+- | |||||||||
| -+ +--+-Xms16090m -Xmx16090m -XX:PermSize=384m -XX:MaxPermSize=384m +- | |||||||||
| -+-XX:NewSize=192m -XX:MaxNewSize=192m -XX:SurvivorRatio=+--+8+- | |||||||||
| -+-XX:+UseLargePages -XX:+UseConcMarkSweepGC +- | |||||||||
| -+-Dsun.rmi.dgc.client.gcInterval=+--+3600000+- | |||||||||
| -+-Dsun.rmi.dgc.server.gcInterval=+--+3600000+- | |||||||||
| -+-Djavax.management.builder.initial=org.jboss.system.server.jmx.MBeanServerBuilderImpl+- | |||||||||
| -+ +--+-Djboss.platform.mbeanserver -Dcom.sun.management.jmxremote +- | |||||||||
| -+-Dcom.sun.management.jmxremote.port=+--+2765+- | |||||||||
| -+-Dcom.sun.management.jmxremote.ssl=+--+false+- | |||||||||
| -+-Dcom.sun.management.jmxremote.authenticate=+ | |||||||||
| -+-XX:+DisableExplicitGC -Djava.endorsed.dirs=/usr/lib/jboss/lib/endorsed+- |
You would want to use this to see what JVM options have or haven’t been specified.
jmap - prints shared object memory maps or heap memory details
Use depends what details of the heap you’d like to collect. For example;
jmap -heap <java PID>will show you a heap summary. Garbage Collector used, heap configuration and generation wise heap usage. You would use this to get a good overview of heap details.
jmap -permstat <java PID>prints permanent generation statistics
| ?Each of these below commands for jmap to output heap dumps will take a long time to run and take alot of disk space on the server |
jmap -heap:format=b <java PID>takes a heap dump in hprof binary format. You would need to use this in diagnosis for memory leaks. The format may be different between java versions could also be in the form of
jmap -dump:live,format=b,file=<filename> <java PID>
jmap -histo <java PID> takes a heap dump and is displayed in a histogram format for each Java class, number of objects, memory size in bytes, and fully qualified class names.
jstat -real time monitoring tool.
jstat -gccause <java PID> <time between samples> <amount of samples>. the -gccause is pretty much the same as gcinfo only this also displays cause of the last and current GC events. As an example;
jstat -gccause 27592 5000 10 will display the info every 5,000 ms and print out 10 times like so.
-+jboss+--+@app010+--+.aci3:~$ jstat -gccause +--+27592+--+5000+--+10+- -+ +--+S0 S1 E O P YGC YGCT FGC FGCT GCT LGCC GCC +- -+ +--+20.06+- -+0.00+- -+85.56+- -+27.76+- -+73.68+- -+35508+- -+526.741+- -+1+- -+10.296+- -+537.036+--+unknown GCCause No GC +- -+ +--+0.00+- -+20.18+- -+0.00+- -+27.77+- -+73.68+- -+35513+- -+526.799+- -+1+- -+10.296+- -+537.094+--+unknown GCCause No GC +- -+ +--+17.95+- -+0.00+- -+43.93+- -+27.77+- -+73.68+- -+35516+- -+526.833+- -+1+- -+10.296+- -+537.129+--+unknown GCCause No GC +- -+ +--+13.35+- -+0.00+- -+2.74+- -+27.77+- -+73.68+- -+35520+- -+526.878+- -+1+- -+10.296+- -+537.174+--+unknown GCCause No GC +- -+ +--+19.66+- -+0.00+- -+0.49+- -+27.78+- -+73.79+- -+35524+- -+526.922+- -+1+- -+10.296+- -+537.218+--+unknown GCCause No GC +- -+ +--+26.85+- -+0.00+- -+79.07+- -+27.78+- -+73.79+- -+35528+- -+526.969+- -+1+- -+10.296+- -+537.265+--+unknown GCCause No GC +- -+ +--+23.75+- -+0.00+- -+32.00+- -+27.78+- -+73.79+- -+35532+- -+527.014+- -+1+- -+10.296+- -+537.309+--+unknown GCCause No GC +- -+ +--+0.00+- -+22.12+- -+67.89+- -+27.78+- -+73.79+- -+35535+- -+527.044+- -+1+- -+10.296+- -+537.340+--+unknown GCCause No GC +- -+ +--+0.00+- -+17.82+- -+75.27+- -+27.79+- -+73.79+- -+35539+- -+527.090+- -+1+- -+10.296+- -+537.386+--+unknown GCCause No GC +- -+ +--+0.00+- -+19.18+- -+1.57+- -+27.80+- -+73.79+- -+35543+- -+527.137+- -+1+- -+10.296+- -+537.432+--+unknown GCCause No GC +- -+jboss+--+@app010+--+.aci3:~$+- |
What the columns mean
| Survivor space 0 utilization as a percentage of the space’s current capacity | Survivor space 1 utilization as a percentage of the space’s current capacity. | Eden space utilization as a percentage of the space’s current capacity. | Old space utilization as a percentage of the space’s current capacity. | Permanent space utilization as a percentage of the space’s current capacity. YGC | Number of young generation GC events. | Young generation garbage collection time. | Number of Full GC events. | Full garbage collection time. | Total garbage collection time. |
Quite a few options can be used to see specific details for young collections, old collections, capacities & more. Recommend reading man page to see full list.
Real case scenarios
Memory leak
- We see that over time on a host the heap memory increases after GC events and doesn’t appear to drop like shown below.
What you’d do in this case is take a few heap dumps on the server right after a Full GC event has cleared objects from the heap. You can take one before the Full GC and after and these can be used by a developer to compare which objects have been cleaned up and which ones have remained. Because some objects may naturally live for a long duration it may take a few heap dumps after Full GC’s every so often to find out the culprits. For this to appear fixed the heap would not look to continually grow over time and will have drops and look more like the graph below.
Garbage Collection doesn’t look to work as expected.
2. A host’s memory looks to reach max and is not reclaiming memory as shown below
Here on the host you would use jstat -gccause <java PID> <time between samples> <amount of samples>. if the output shows many Full GC collections taking place this is most likely due to a memory leak again, as each Full GC is only able to collect some data and executes again as it is near full. The underlying issue would most likely be a memory leak again.