Neo4J APOC plugin (3.1.3.6) is very slow

Question

I recently upgraded my Neo4j to 3.1.3, and with it I got the latest APOC plugin (3.1.3.6).

I had some code that worked fine, and could create ~ 3 million relationships in about a minute and half the time of the wall. But now it works for more than 8 hours and does not show any signs of a stop ...

Since the code used to run without any problems, I hope that something should change between the versions that led to my code being processed.

Should rock_n_roll have to be changed (maybe apoc.periodic.commit with positional arguments or something else)? Thank you for understanding.

This is what I am running.

 CALL apoc.periodic.rock_n_roll( "MATCH (c:ChessPlayer),(r:Record) WHERE c.ChessPlayer_ID = r.ChessPlayer RETURN c,r", "CYPHER planner=rule WITH {c} AS c, {r} AS r CREATE (c)-[:HAD_RECORD]->(r)", 200000) 

Answer 1

I understand that a call requests the Cartesian product of ChessPlayers and Records, and then tries to filter them out successively by line, and then performs a batch update of these final results (which eats a lot of memory, I think this is one opening operation - this is what kills you ) Therefore, if you can break it down so that each transaction can touch as many nodes as possible, it should be able to perform much better (especially if r.ChessPlayer is indexed, since now you do not need to download all of them)

 CALL apoc.periodic.rock_n_roll( "MATCH (c:ChessPlayer) WHERE NOT EXISTS((c)-[:HAD_RECORD]->()) RETURN c", "MATCH (r:Record) WHERE c.ChessPlayer_ID = r.ChessPlayer WITH c,r CREATE UNIQUE (c)-[:HAD_RECORD]->(r)", 100000) 

period.commit () will work on a similar principle. The smaller (the least nodes are affected), you can make each transaction, the faster the package will be.