Neo4jClient - merging inside ForEach with 1000 very slow (unique limitation)

This is related to this SO question: Merge card options or creating a batch operation through the client?

This request:

FOREACH (n in {set} | MERGE (c:Label {Id : n.Id}) SET c = n) 

Works well for updating or creating new nodes based on a unique key (in this case, Id). It is, however, very slow. When I try to process a list of 42,000 items, in batches of 1000, it will take about 2 minutes. As for the hardware, this works on my laptop (i7, 8 GB of RAM, Samsung 840 Pro SSD). By the way, this works on a clean database, so initially there are no matches, just created. (Although I want it to support data updates, hence the use of Merge.

C # code:

        createUniqueConstraint(label, PK_Field); //Creating a constraint automatically creates an index
        string propKey = label + "s" ;
        string strForEach = string.Format("(n in {{{0}}} |  MERGE (c:{1} {{{2} : n.{2}}}) SET c = n)",
            propKey, label, PK_Field);
        foreach (var entities in list.Batch(1000))
        {
            var query = client
                            .Cypher
                            .ForEach(strForEach)
                            .WithParam(propKey, entities.ToList());

            query.ExecuteWithoutResults();
            Debug.WriteLine("Batch passed for " + label + " ("+entities.Count()+" items)");
        }

UPDATE

node, Merge foreach ( ), CPU 32%, 40-50 . 50 . 2 .

( , , 2 )

QueryText ( 42k)

[Neo4jClient.Cypher.CypherFluentQuery] = 
"FOREACH (n in System.Collections.Generic.List`1[Common.Models.Contact] |
 MERGE (c:Contact {ContactId : n.ContactId}) SET c = n)"

: 655 seconds to process 42k items

+1

Source: https://habr.com/ru/post/1535452/


All Articles