Memory for Java application: How much is enough

In my current project we designed a Java application which has high memory requirements.

We have been using open jdk11, which has some scope of improvement when it comes to grabage collection and memory optimization.

It needed 60 GB heap to handle extra load coming during peak period we provided 80 GB heap memory. Total memory. provided to machine was 100 GB. Even after providing this much extra memory we started getting alerts in production about memory usage for a machine being very high like 90% and more. This is was not expected considering we gave enough memory to the machine.

So after lot of analysis how memory is used in a machine here is what we concluded.

Memory across processes in a system

Linux will divide the memory among processes and when you run top command you get below kind of output:

As per this we have 2 kinds of memory for every process

• Virtual memory(Indicated as VIRT): Virtual memory is the memory which process can end up using. Kind of upper limit on memory usage of process. We observed this changing little bit over a period of time.
• Resident memory(Indicated as RES): Resident memory is the memory which process is using currently.

Sometime sum of virtual memory across all processes running into system could be larger than total memory in system, which indicates if some important process needs more memory then OS can shut down processes which it consider not so important to provide memory to the processes which OS thinks are more important.

Mostly Java applications will come into category of not so important process for Operating system.

Below are the diagrams which I used to explain this whole memory concept to other folks.

Memory allocation within Java process

When someone talks about memory in Java it usually refers heap memory which is 80% of memory, but Java has 2 kinds of memory:

1. Heap memory: Heap memory is the run time data area from which the memory for all java class instances and arrays is allocated. The heap is created when the Java Virtual Machine starts up and may increase or decrease in size while the application runs. The size of the heap can be specified using –Xms VM option. The heap can be of fixed size or variable size depending on the garbage collection strategy. Maximum heap size can be set using –Xmx option.By default, the maximum heap size if not specifies will be 25% of total memory of machine
2. Non heap memory/Native memory: The Java Virtual Machine has memory other than the heap, referred to as Non-Heap Memory. It is created at the JVM startup and stores per-class structures such as runtime constant pool, field and method data, and the code for methods and constructors, as well as interned Strings. The default maximum size of non-heap memory is 64 MB. This can be changed using –XX:MaxPermSize VM option.

Most of the application during deployment would have heap size defined but ignore non heap memory considering it will be very negligible.

Where we went wrong in calibrating memory

1. Non-heap memory requirement: When we ran top command for our application it returned 88 GB, which was very surprising as heap will never grow beyond 80 GB, which made us curious to know more about non heap memory. So its important to know non heap memory requirment as well. You need to do 2 steps for it.

• Add -XX:NativeMemoryTracking=summary flag while starting application.
• To know the size of non heap memory in use run jcmd <pid> VM.native_memory summary

After running this we realised that non-heap memory for our application is around 8 GB. This is huge considering our machine total memory was only 100 GB, out of which 88GB was only used by single Java process and not leaving enough space for other processes. Biggest contributor in non-heap memory was garbage metadata and I assumed that as heap size was very high so GC will also take good amount of memory to keep meta data. I am yet to do more analysis on this further.

2. Active vs passive heap usage: We were using app dynamics as monitoring tool. As per app dynamics heap usage for our application was only 25–30 GB but total process has been using 87GB memory. After reading more about heap memory we figured out the concept of active vs passive heap memory. Till Java11, once heap acquires a space in memory, it is not guarantee that Java process will release that space back if not in use actively, so even is once process end us using heap of 60 GB, it can hold that up 60 GB till end even its not using it. Java 17 has done improvement over this behaviour where process will release memory back to OS.

3. Other Monitoring process in system: As we were using app dynamics for monitoring, we were running app dynamics agent in the machine which is also a Java process, we did not mention -Xms and -Xmx for this agent, which means if required its heap memory can keep growing will 25 GB. Hence realised that any Java process in a machine needs to have Xms and Xmx defined so that it stays in that boundary and that needs to be considered while deciding memory requirement for machine.

My journey of understanding memory for Java has just started hope to write more blogs in near future.