Neo4j and Hugepages

Question

Since Neo4j works mostly in memory, I was wondering if it would be useful to include huge pages ( https://www.kernel.org/doc/Documentation/vm/hugetlbpage.txt ) in my Linux kernel, and then XX: + UseLargePages or maybe -XX: + UseHugeTLBFS in the (OpenJDK 8) JVM?

If so, what rule of thumb should be used to determine how many huge pages are configured?

Neo4j performance guide ( http://neo4j.com/docs/stable/performance-guide.html ) does not mention this, and Google did not disclose to anyone else its discussion (in the first pair of search pages anyway), therefore I thought I would ask.

I am struggling to get acceptable performance from my new copy of Neo4j (2.3.2-community). Any bit will help. I want to know if it is worth a try before I reset the database for changing JVM flags ... I hope someone else will do some experiments in this direction.

Thanks!

Answer 1

Since Neo4j makes its own swap file and does not rely on the OS to do this, it should be profitable or at least not hurt. Huge pages will reduce the chance of TLB cache misses when you use a large amount of memory, which Neo4j would often want to do when it contained a lot of data.

However, Neo4j does not directly use huge pages, although this is possible, and that would be a nice addition. This means that you have to rely on transparent huge pages and any features provided by the JVM. Huge transparent pages can create more or less short stalls when combining small pages.

If you have a representative staging environment, then I advise you to first make changes and measure their effect.

Transparent huge pages are mostly a problem for programs using mmap , because I think it can lead to a change in the size of the I / O unit, which will make the latency with the hard page much higher. I'm not quite sure about this, so please correct me if I am wrong.

The JVM actually uses mmap for telemetry and instrumentation via a file in /tmp , so make sure this directory is set to tmpfs to avoid gnarly IO stalls, for example, during safe points (!!!). Always do this, even if you are not using huge pages.

Also make sure you are using the latest Linux kernel and the latest version of Java.

You can squeeze some percentage points out of it with the G1 setting, but it's a bit of a black art.