Is Java's HashMap performance 25% less than Java 8?

Question

Note. This issue is not about a performance issue. I only observe a difference in performance that I cannot explain / understand.

Comparing newly developed code designed for Java 9, I found something strange. A (very) simple 5-key HashMap test shows that Java 9 is much slower than Java 8. Could this be explained or is my (control) code just wrong?

code:

 @Fork( jvmArgsAppend = {"-Xmx512M", "-disablesystemassertions"} ) public class JsonBenchmark { @State(Scope.Thread) public static class Data { final static Locale RUSSIAN = new Locale("ru"); final static Locale DUTCH = new Locale("nl"); final Map<Locale, String> hashmap = new HashMap<>(); public Data() { hashmap.put(Locale.ENGLISH, "Flat flashing adjustable for flat angled roof with swivel"); hashmap.put(Locale.FRENCH, "Solin pour toit plat inclinée"); hashmap.put(Locale.GERMAN, "Flachdachkragen Flach Schrägdach"); hashmap.put(DUTCH, "Plakplaat vlak/hellend dak inclusief glijschaal"); hashmap.put(RUSSIAN, "    .  . "); } } @Benchmark public int bmHashMap(JsonBenchmark.Data data) { final Map<Locale, String> m = data.hashmap; int sum = 0; sum += m.get(Data.RUSSIAN).length(); sum += m.get(Locale.FRENCH).length(); sum += m.get(Data.DUTCH).length(); sum += m.get(Locale.ENGLISH).length(); sum += m.get(Locale.GERMAN).length(); return sum; } } 

Results:

• Java 8_151: JsonBenchmark.bmHashMap thrpt 40 47948546.439 ± 560763.711 ops / s
• Java 9_181: JsonBenchmark.bmHashMap thrpt 40 34962904.479 ± 276045.691 ops / s ( - / - 27% !)

UPDATE

Thanks for the answers and great comments.

• Suggestion from @Holger. My first reaction was: This should be an explanation. However, if I test only the String#length() function, then there is no significant difference in performance. And, when I only focus on the HashMap#get() methods (as suggested by @Eugene), there is still a 10-12% difference.

• Suggestion from @Eugene. I changed the parameters (more iterations of warming up, more memory), but I can not reproduce your result. However, I increased the heap to 4G. But that can't explain the difference, right?

• Suggestion from @Alan Bateman. Yes, it improves performance! However, the difference is about 20%.

Answer 1

You are testing more than just a HashMap . Not only do you call HashMap.get , you implicitly call Locale.hashCode and Locale.equals . In addition, you call String.length .

Now all four could change their performance characteristics, so you will need much more tests to determine which method demonstrates different performance.

But the hottest candidate is String.length . In Java 9, the String class no longer uses the char[] array, but the byte[] array to encode Latin 1 strings using only one byte per character, which significantly reduces the memory footprint of typical applications. This, however, implies that the length does not always match the length of the array. Thus, the complexity of this operation has changed.

But keep in mind that your result is about 77 ns in microbusiness. This is not enough to evaluate the impact on a real application ...

Answer 2

I had a hint on jmh setup, moreover, it was about HashMap . As already noted, you measure a lot more than just HashMap::get here. But even in this case, I doubted that java-9 would be much slower, so I measured myself (the last jmh is built from sources, java-8 and 9).

I did not change my code - I just added more heaps (10 GB) and even more workouts, thereby reducing the "error" that you see after ±

Using java-8:

 Benchmark Mode Cnt Score Error Units SOExample.bmHashMap avgt 25 22.059 ± 0.276 ns/op 

Using java-9:

 Benchmark Mode Cnt Score Error Units SOExample.bmHashMap avgt 25 23.954 ± 0.383 ns/op 

The results look at the level with almost no noticeable difference (it's still nano-seconds), as you can see. Also, if you really want to test only HashMap::get , than your methods could just return a call to this, for example:

 @Benchmark @Fork(5) public int bmHashMap(SOExample.Data data) { return data.hashmap.get(data.key); // where key is a random generated possible key }