Does Neo4J 3.x use more memory than Neo4J 2.x? How to avoid this?

Question

I upgraded my graphical DB from Neo4J 2.0.4 to Neo4J 3.3.3, and when I run it with my application, it now uses twice as much memory ( java on my Mac) than before. (I am running Java 1.8 on my Mac)

When I launch Neo4J 2, it uses about 250 MB for the same tasks and requests. But Neo4J 3 uses about 500 MB.

I thought updates should be more efficient?

What would be a possible way to reduce memory usage?

Answer 1

Neo4j is based on the JVM, its memory depends on the size of the heap.

If you have not configured your size, Neo4j has heursitic to calculate it, and by the time this heuristic has changed.

Neo4j 3.X has 3 memory spaces:

a bunch of jvm

The heap is used to store all transaction data, block nodes and relationships, and also to cache the query execution plan.

You can adjust its size in conf/neo4j.conf :

 # Java Heap Size: by default the Java heap size is dynamically # calculated based on available system resources. # Uncomment these lines to set specific initial and maximum # heap size. dbms.memory.heap.initial_size=512m dbms.memory.heap.max_size=512m 

Page Cache

The database does a lot of disk I / O. To be fast, he needs to put his data in RAM. Neo4j 3.X does this using the pagecache mechanism, which places some parts of its binary data files into RAM. Therefore, your most frequently used data is in RAM.

This can be configured in conf/neo4j.conf :

 # The default page cache memory assumes the machine is dedicated to running # Neo4j, and is heuristically set to 50% of RAM minus the max Java heap size. dbms.memory.pagecache.size=10g 

Indices

Indexes are used to quickly search for nodes for your queries, but if these indexes are not in RAM, they will be slow.

It is not possible to configure this memory array in Neo4j, but with rule three it is easy to calculate it (Server RAM = Heap + pagecache + indexes + some free memory for the OS).

You can calculate the size of your indexes with this command:

 $> du -sh data/databases/graph.db/indexes 

Conclusion

Since Neo4j 2, there are many improvements (functionality, features), so yes, it is more efficient.

I don’t know a more effective way to print fingers on RAM, I have never done such a test, but RAM is now cheap.

The only thing I can tell you is that you need a smaller heap, now the data cache is disabled. In 2.X, Neo4j saved a graph in memory for quick access.

But Neo4j is very configurable, so if you have a tiny server and you want to fine tune memory consumption, you need to set the heap and pagecache sizes.