Effectively find and replace lines in documents

Question

I have the following query: find tags   in the name field and replace them with empty space - to get rid of them.
Name strings can have from 1 to many   , eg.

 AA aa AA  aa AA   aa AA    aa AA AA aaaaaaaa 

... like this.

  db.tests.find({'name':/.* .*/}).forEach(function(test){ test.name = test.name.replace(" ",""); db.tests.save(test); }); db.tests.find({'name':/.*  .*/}).forEach(function(test){ test.name = test.name.replace("  ",""); db.tests.save(test); }); db.tests.find({'name':/.*   .*/}).forEach(function(test){ test.name = test.name.replace("   ",""); db.tests.save(test); }); 

Besides repeating the same query pattern, is there a better solution for handling this scenario in terms of less duplication and better performance?

Answer 1

Of course, if all you want to do is delete the   from your text, then you just do a global match and replace:

 db.tests.find({ "name": /\ /g }).forEach(function(doc) { doc.name = doc.name.replace(/ /g,""); db.tests.update({ "_id": doc._id },{ "$set": { "name": doc.name } }); }); 

Therefore, it should not be necessary to write out each combination; the regular expression will replace the very match with the /g option. It is also possible to use /m for a multi-line line. The string "name" contains newline characters. See The basic regexer example.

It is also recommended that you use $set to change only those fields that you really want, and not .save() entire document back. There is less traffic and less chance to overwrite changes that might have been made by another process since reading the document.

Ideally, you would use the Bulk Operations API with MongoDB versions 2.6 and higher. This allows package updates, so again less traffic occurs between the client and server:

 var bulk = db.tests.initializeOrderedBulkOp(); var count = 0; db.tests.find({ "name": /\ /g }).forEach(function(doc) { doc.name = doc.name.replace(/ /g,""); bulk.find({ "_id": doc._id }) .updateOne({ "$set": { "name": doc.name } }); count++; if ( count % 1000 == 0 ) { bulk.execute(); bulk = db.tests.initializeOrderedBulkOp(); } }); if ( count % 1000 != 0 ) bulk.execute(); 

These are your main ways to improve this. Unfortunately, for the MongoDB update statement, there is no way to use the existing value as part of its expression to update this way, so the only way is to loop, but you can do a lot to reduce the operations as shown.