MongoDB dot (.) By Key Name

Question

It seems that the mango does not allow you to enter keys with a period (.) Or dollar sign ($), however, when I imported the JSON file containing the period in it using the mongoimport tool, it worked fine. The driver complains about trying to insert this item.

Here's what the document looks like in the database:

{ "_id": { "$oid": "..." }, "make": "saab", "models": { "9.7x": [ 2007, 2008, 2009, 2010 ] } } 

Am I doing all this wrong and should not use hash maps similar to those with external data (i.e. models), or can I somehow escape the point? Maybe I think too much about Javascript.

Answer 1

MongoDB does not support keys with a dot in them, so you have to pre-process your JSON file to remove / replace them before importing or you will "I will configure myself for all kinds of problems.

There is no standard solution to this problem; the best approach is too dependent on the specifics of the situation. But, if possible, I avoid any encoder / decoder approach, as you will continue to tirelessly perform the inconvenience when the JSON restructuring is supposed to be one-time.

Answer 2

Mongo docs suggest replacing illegal characters like $ and . their equivalents in unicode format.

In these situations, the keys will need to replace the reserved $ and. characters. Any character is sufficient, but consider the use of Unicode full-width equivalents: U + FF04 (ie, "$") and U + FF0E (i.e., ".").

Answer 3

As mentioned in other answers, MongoDB does not allow $ or . characters as card keys due to restrictions on field names . However, as mentioned in the Dollar Sign Operator, the exclusion of this restriction does not prevent you from inserting documents with such keys, it simply prohibits you from updating or requesting them.

The problem is a simple replacement . with [dot] or U+FF0E (as mentioned elsewhere on this page), what happens when a user legitimately wants to save the [dot] key or U+FF0E ?

The approach used by the Fantom afMorphia driver is to use unicode escape sequences similar to Java, but you must first escape the escape character. Essentially, the following line replacements (*) are performed:

 \ --> \\ $ --> \u0024 . --> \u002e 

Reverse replacement is performed when the card keys are subsequently read from MongoDB.

Or in Fantom code:

 Str encodeKey(Str key) { return key.replace("\\", "\\\\").replace("\$", "\\u0024").replace(".", "\\u002e") } Str decodeKey(Str key) { return key.replace("\\u002e", ".").replace("\\u0024", "\$").replace("\\\\", "\\") } 

The only time a user needs to know about such conversions is to create queries for such keys.

Considering that it is customary to store dotted.property.names in databases for configuration purposes, I believe that this approach is preferable to simply banning all such card keys.

(*) afMorphia actually follows the full / correct Unicode escaping rules , as mentioned in Unicode escaping syntax in Java, but the replacement sequence described works just as well.

Answer 4

The solution that I just implemented that I am really happy with is to split the name and value of the key into two separate fields. That way, I can keep the characters exactly the same and not worry about any of these syntax nightmares. The document will look like this:

 { ... keyName: "domain.com", keyValue: "unregistered", ... } 

You can still query for this quite simply by simply doing find in the keyName and keyValue fields .

Therefore, instead of:

  db.collection.find({"domain.com":"unregistered"}) 

which doesn't actually work as expected, you run:

 db.collection.find({keyName:"domain.com", keyValue:"unregistered"}) 

and it will return the expected document.

Answer 5

The latest stable version (v3.6.1) MongoDB now supports periods (.) In key or field names.

Field names can contain periods (.) And dollars ($).

Answer 6

You can try using a hash in the key instead of a value, and then store that value in a JSON value.

 var crypto = require("crypto"); function md5(value) { return crypto.createHash('md5').update( String(value) ).digest('hex'); } var data = { "_id": { "$oid": "..." }, "make": "saab", "models": {} } var version = "9.7x"; data.models[ md5(version) ] = { "version": version, "years" : [ 2007, 2008, 2009, 2010 ] } 

You will then access the models using the hash later.

 var version = "9.7x"; collection.find( { _id : ...}, function(e, data ) { var models = data.models[ md5(version) ]; } 

Answer 7

From MongoDB docs "the.". the character should not appear anywhere in the key name. "It looks like you have to come up with a coding scheme or get around.

Answer 8

Late answer, but if you use Spring and Mongo, Spring can control the conversion for you using MappingMongoConverter . This solution is from JohnnyHK, but is handled by Spring.

 @Autowired private MappingMongoConverter converter; @PostConstruct public void configureMongo() { converter.setMapKeyDotReplacement("xxx"); } 

If your saved Json:

 { "axxxb" : "value" } 

Via Spring (MongoClient) this will read like:

 { "ab" : "value" } 

Answer 9

You will need to avoid the keys. Since it seems that most people do not know how to avoid strings correctly, follow these steps:

• select an escape character (it’s best to choose a character that is rarely used). For example. '~'
• To escape, first replace all instances of the escape character with some sequence added by your escape character (e.g. '~' → '~ t'), then replace any character or sequence that you need to execute with some sequence added with using your escape. For example. '' → '~ p'
• To unescape, first remove the escape sequence from the entire instance of your second escape sequence (for example, '~ p' → '.'), Then convert the sequence of escape characters to a single escape character (for example, '~ s' →' ~ ')

Also, remember that mongo also does not allow keys to start with '$', so you need to do something like that there

Here is the code that does this:

 // returns an escaped mongo key exports.escape = function(key) { return key.replace(/~/g, '~s') .replace(/\./g, '~p') .replace(/^\$/g, '~d') } // returns an unescaped mongo key exports.unescape = function(escapedKey) { return escapedKey.replace(/^~d/g, '$') .replace(/~p/g, '.') .replace(/~s/g, '~') } 

Answer 10

I use the following JavaScript escaping for each object key:

 key.replace(/\\/g, '\\\\').replace(/^\$/, '\\$').replace(/\./g, '\\_') 

I like that it only replaces $ at the beginning and does not use Unicode characters, which can be difficult to use in the console. _ for me is much more readable than the Unicode character. It also does not replace one set of special characters ( $ ,.) With another (unicode). But correctly escapes with the traditional \ .

Answer 11

Not perfect, but will work in most situations: replace the forbidden characters with something else. Since this is in the keys, these new characters should be fairly rare.

 /** This will replace \ with ⍀, ^$ with '₴' and dots with ⋅ to make the object compatible for mongoDB insert. Caveats: 1. If you have any of ⍀, ₴ or ⋅ in your original documents, they will be converted to \$.upon decoding. 2. Recursive structures are always an issue. A cheap way to prevent a stack overflow is by limiting the number of levels. The default max level is 10. */ encodeMongoObj = function(o, level = 10) { var build = {}, key, newKey, value //if (typeof level === "undefined") level = 20 // default level if not provided for (key in o) { value = o[key] if (typeof value === "object") value = (level > 0) ? encodeMongoObj(value, level - 1) : null // If this is an object, recurse if we can newKey = key.replace(/\\/g, '⍀').replace(/^\$/, '₴').replace(/\./g, '⋅') // replace special chars prohibited in mongo keys build[newKey] = value } return build } /** This will decode an object encoded with the above function. We assume the structure is not recursive since it should come from Mongodb */ decodeMongoObj = function(o) { var build = {}, key, newKey, value for (key in o) { value = o[key] if (typeof value === "object") value = decodeMongoObj(value) // If this is an object, recurse newKey = key.replace(/⍀/g, '\\').replace(/^₴/, '$').replace(/⋅/g, '.') // replace special chars prohibited in mongo keys build[newKey] = value } return build } 

Here is the test:

 var nastyObj = { "sub.obj" : {"$dollar\\backslash": "$\\.end$"} } nastyObj["$you.must.be.kidding"] = nastyObj // make it recursive var encoded = encodeMongoObj(nastyObj, 1) console.log(encoded) console.log( decodeMongoObj( encoded) ) 

and results - note that the values ​​do not change:

 { sub⋅obj: { ₴dollar⍀backslash: "$\\.end$" }, ₴you⋅must⋅be⋅kidding: { sub⋅obj: null, ₴you⋅must⋅be⋅kidding: null } } [12:02:47.691] { "sub.obj": { $dollar\\backslash: "$\\.end$" }, "$you.must.be.kidding": { "sub.obj": {}, "$you.must.be.kidding": {} } } 

Answer 12

Lodash parties let you change

 { 'connect.sid': 's:hyeIzKRdD9aucCc5NceYw5zhHN5vpFOp.0OUaA6' } 

at

 [ [ 'connect.sid', 's:hyeIzKRdD9aucCc5NceYw5zhHN5vpFOp.0OUaA6' ] ] 

using

 var newObj = _.pairs(oldObj); 

Answer 13

You can save it as is and convert it to beautiful after

I wrote this example in Livescript. You can use livescript.net to rate it.

 test = field: field1: 1 field2: 2 field3: 5 nested: more: 1 moresdafasdf: 23423 field3: 3 get-plain = (json, parent)-> | typeof! json is \Object => json |> obj-to-pairs |> map -> get-plain it.1, [parent,it.0].filter(-> it?).join(\.) | _ => key: parent, value: json test |> get-plain |> flatten |> map (-> [it.key, it.value]) |> pairs-to-obj 

He will produce

 {"field.field1":1, "field.field2":2, "field.field3":5, "field.nested.more":1, "field.nested.moresdafasdf":23423, "field3":3} 

Answer 14

Give me my advice: you can use JSON.stringify to save an object / array containing the key name with dots, then parse the Object string with JSON.parse to handle when to get data from the database

Another workaround: Rebuild your circuit as follows:

 key : { "keyName": "ab" "value": [Array] } 

Answer 15

For PHP, I am replacing the HTML value for this period. This is "." .

It is stored in MongoDB as follows:

  "validations" : { "4e25adbb1b0a55400e030000" : { "associate" : "true" }, "4e25adb11b0a55400e010000" : { "associate" : "true" } } 

and PHP code ...

  $entry = array('associate' => $associate); $data = array( '$set' => array( 'validations.' . str_replace(".", `"."`, $validation) => $entry )); $newstatus = $collection->update($key, $data, $options); 

Answer 16

The latest MongoDB supports dotted keys, but the java MongoDB driver does not. Therefore, to make it work in Java, I pulled the code from github repo java-mongo-driver and made the appropriate changes to their isValid Key function, created a new jar from it, using it now.

Answer 17

Replace the period ( . ) Or dollar ( $ ) with other characters that will never be used in the actual document. And restore the point ( . ) Or dollar ( $ ) upon receipt of the document. The strategy will not affect the data that the user reads.

You can select a character from all characters .

Answer 18

The strange thing is that using mongojs I can create a document with a dot if I set _id myself, however I cannot create a document when _id is generated:

Works:

 db.testcollection.save({"_id": "testdocument", "dot.ted.": "value"}, (err, res) => { console.log(err, res); }); 

Does not work:

 db.testcollection.save({"dot.ted": "value"}, (err, res) => { console.log(err, res); }); 

At first I thought that updating a document with a dot key also works, but that means the dot is a subkey!

Seeing how mongojs processes the point (subkey), I'm going to make sure that my keys do not contain the point.

Answer 19

Now supported

MongoDb 3.6 and later supports both points and dollars in field names. See below JIRA: https://jira.mongodb.org/browse/JAVA-2810

Upgrading your Mongodb to 3.6+ sounds like the best way.

Answer 20

/home/user/anaconda3/lib/python3.6/site-packages/pymongo/collection.py

This is detected in error messages. If you use anaconda (find the appropriate file, if not), just change the value from check_keys = True to False in the above file. It will work!