How to really compare the memory usage of a Java application

Question

I want to compare different implementations of Java programs in terms of memory efficiency. There are various usage scenarios formulated as JUnit test cases. In fact, all code is open source: https://github.com/headissue/cache2k-benchmark

The general wisdom to use in the used memory of a Java program is this: of Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()course, you can also use the JMX interface to get these values.

However, certain values ​​of the used memory are not reliable. Possible reasons:

• May be irrevocable trash
• Fragmentation exists if GC did not compact.

So far, I have been experimenting with switching to a serial GC and forcing garbage collection using Runtime.getRuntime (). gc () before reading the value. I put the experimental code for this: https://github.com/cruftex/java-memory-benchmark

If I make three calls to gc before reading the values, I get this output ( mvn test | grep loopCountusing jdk1.7.0_51):

testBaseline1: used=1084168, loopCount=0, total=124780544
testBaseline2: used=485632, loopCount=0, total=124780544
testBaseline3: used=483760, loopCount=0, total=124780544
testBaseline4: used=483800, loopCount=0, total=124780544
testBaseline: used=484160, loopCount=0, total=124780544
test100MBytes: used=105341496, loopCount=0, total=276828160
test127MBytes: used=133653088, loopCount=0, total=469901312
test27MBytes: used=28795528, loopCount=0, total=317755392
test10MBytes: used=10969776, loopCount=0, total=124784640

With four gc calls (as noted) I get:

testBaseline1: used=483072, loopCount=0, total=124780544
testBaseline2: used=483728, loopCount=0, total=124780544
testBaseline3: used=483768, loopCount=0, total=124780544
testBaseline4: used=483808, loopCount=0, total=124780544
testBaseline: used=483848, loopCount=0, total=124780544
test100MBytes: used=105341504, loopCount=0, total=276828160
test127MBytes: used=133653096, loopCount=0, total=469901312
test27MBytes: used=28795536, loopCount=0, total=139239424
test10MBytes: used=10969784, loopCount=0, total=124784640

So, it is empirically shown t that with four calls to GC the correct seam of the results will be correct. From the output of the GC statistics, I see that the first GC fills the space, and the fourth GC call reduces it:

2015-01-08T02:30:35.069+0100: [Full GC2015-01-08T02:30:35.069+0100: [Tenured: 0K->1058K(83968K)
2015-01-08T02:30:35.136+0100: [Full GC2015-01-08T02:30:35.136+0100: [Tenured: 1058K->1058K(83968K)
2015-01-08T02:30:35.198+0100: [Full GC2015-01-08T02:30:35.198+0100: [Tenured: 1058K->1058K(83968K)
2015-01-08T02:30:35.263+0100: [Full GC2015-01-08T02:30:35.264+0100: [Tenured: 1058K->471K(83968K)

Final code to get memory usage value:

try {
  Runtime.getRuntime().gc();
  Thread.sleep(55);
  Runtime.getRuntime().gc();
  Thread.sleep(55);
  Runtime.getRuntime().gc();
  Thread.sleep(55);
  Runtime.getRuntime().gc();
  Thread.sleep(55);
} catch (Exception ignore) { }
long _usedMem;
long _total;
long _total2;
long _count = -1;
// loop to get a stable reading, since memory may be resized between the method calls
do {
  _count++;
  _total = Runtime.getRuntime().totalMemory();
  try {
    Thread.sleep(12);
  } catch (Exception ignore) { }
  long _free = Runtime.getRuntime().freeMemory();
  _total2 = Runtime.getRuntime().totalMemory();
  _usedMem = _total - _free;
} while (_total != _total2);
System.out.println(_testName + ": used=" + _usedMem + ", loopCount=" + _count + ", total=" + _total);

I am pretty unsure that this approach is constantly creating reliable results. So, some questions:

• - Java?
• , ( ) GC ?
• , GC? (BTW: java 8 )
• JVM: " , "?
• , " " ?

Update:

, , GC, . . .

Answer 1

, - .

, , JVM , , :

GcFinalization.awaitFullGc();

Guava, Maven :

 <dependency>
    <groupId>com.google.guava</groupId>
    <artifactId>guava-testlib</artifactId>
    <version>18.0</version>
</dependency>

:

public static void awaitFullGc() {
   final CountDownLatch finalizerRan = new CountDownLatch(1);
   WeakReference<Object> ref = new WeakReference<Object>(
      new Object() {
         @Override protected void finalize() { finalizerRan.countDown(); }
      });

   await(finalizerRan);
   awaitClear(ref);

   // Hope to catch some stragglers queued up behind our finalizable object
   System.runFinalization();
 }

( ThreadMXBean) - ( System.currentTimeMills). , , .

Answer 2

-, JMH, , Java-.

Runtime.getRuntime().gc() - , GC. , GC, GC.

, GC, ~ 4 GC. GC - . JMH, - , 10 ...

, JMH-like (~ 30 ), GC GC ... , - .

Answer 3

Java- .

- :

-Xloggc:gc.log_impl1 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps

2

-Xloggc:gc.log_impl2 -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps

Then download HPjmeter , download both files to the console and use the gc comparison function. The graphs may be a little skewed, but you will get a pretty good idea of ​​how the memory profiles of programs differ.

I would not synthesize GC synthetically.