mongoimport of a JSON Array with ‘to’ field got issues ?!!

While importing attached JSON array as below, I see below errors –

Sample records from the JSON file are below.

[{ “_id” : { “$oid” : “52fa519da6342a0dcd97222a” }, “active” : true, “from” : { “$date” : “2000-01-01T00:00:00.000-0500” }, “kind” : “Server”, “name” : “EmployeeSearchMaxLimit”, “to” : { “$date” : “2014-02-26T14:45:03.133-0500” }, “type” : “String”, “value” : “20” },

{ “_id” : { “$oid” : “52fa519da6342a0dcd97222b” }, “active” : true, “from” : { “$date” : “2000-01-01T00:00:00.000-0500” }, “kind” : “Server”, “name” : “ExpireInMonths”, “to” : { “$date” : “2014-02-26T14:45:03.439-0500” }, “type” : “Double”, “value” : 3 },

{ “_id” : { “$oid” : “52fa519da6342a0dcd97222c” }, “active” : true, “from” : { “$date” : “2000-01-01T00:00:00.000-0500” }, “kind” : “Server”, “name” : “From”, “to” : { “$date” : “2014-02-26T14:45:03.591-0500” }, “type” : “String”, “value” : “dbversity@gmail.com” },

{ “_id” : { “$oid” : “52fa519da6342a0dcd97222d” }, “active” : true, “from” : { “$date” : “2000-01-01T00:00:00.000-0500” }, “kind” : “Server”, “name” : “FulfillmentBou”, “to” : { “$date” : “2014-02-26T14:45:03.743-0500” }, “type” : “String”, “value” : “ou=infra,dc=dbverst,dc=dbversity,dc=com” },

{ “_id” : { “$oid” : “52fa519da6342a0dcd97222e” }, “active” : true, “from” : { “$date” : “2000-01-01T00:00:00.000-0500” }, “kind” : “Server”, “name” : “FulfillmentEnabled”, “to” : { “$date” : “2014-02-26T14:45:03.893-0500” }, “type” : “Boolean”, “value” : true } ]

[ root @ dbversity : /opt/mongodb/bin ] ll -lhtr /tmp/appConfig2_new.json 
– 1 dbversit mongod 156K May 16 05:58 /tmp/appConfig2_new.json
[ root @ dbversity : /opt/mongodb/bin ] 
[ root @ dbversity : /opt/mongodb/bin ] dos2unix /tmp/appConfig2_new.json 
dos2unix: converting file /tmp/appConfig2_new.json to UNIX format …
[ root @ dbversity : /opt/mongodb/bin ] 
[ root @ dbversity : /opt/mongodb/bin ] wc -l /tmp/appConfig2_new.json 
1 /tmp/appConfig2_new.json
[ root @ dbversity : /opt/mongodb/bin ]

[ root @ dbversity : /opt/mongodb/bin ] ./mongoimport -d newdb -c newcol -u admin -p pwd –authenticationDatabase admin –file /tmp/appConfig2_new.json –drop –jsonArray -vvvvv
2016-05-24T08:02:21.388-0400 creating new connection to:127.0.0.1:27017
2016-05-24T08:02:21.388-0400 [ConnectBG] BackgroundJob starting: ConnectBG
2016-05-24T08:02:21.388-0400 connected to server 127.0.0.1:27017 (127.0.0.1)
2016-05-24T08:02:21.388-0400 connected connection!
connected to: 127.0.0.1
2016-05-24T08:02:21.389-0400 ns: newdb.newcol
2016-05-24T08:02:21.389-0400 dropping: newdb.newcol
2016-05-24T08:02:21.389-0400 filesize: 159522
2016-05-24T08:02:21.395-0400 warning: log line attempted (142k) over max size (10k), printing beginning and end … User Assertion: 13293:Invalid JSON passed to mongoimport: code FailedToParse: FailedToParse: Expecting ‘

{‘: offset:0 of:}

,,

——- all JSON file data here ————

2016-05-24T07:56:06.692-0400 check 9 37
2016-05-24T07:56:06.692-0400 imported 37 objects
encountered 1 error(s)
[ root @ dbversity : /opt/mongodb/bin ]

However, the JSON file is valid, version 2.6 of mongoimport is having difficulty parsing the “to” field in documents such as:

The combination of $date and $numberLong is a stretch for it to parse. The good news is that version 3.0 of mongoimport doesn’t have this limitation.

As such, you can connect version 3.0 of mongoimport to a version 2.6 mongod to import the data.

How to identify the issue ?

The error messages from mongoimport were cryptic, and confusing. So, I had to fall back to plain old detective work. I started with the error message :

2016-05-24T07:56:06.692-0400 imported 37 objects

to determine the first document that failed. From there, I put the one document that failed in a file by itself, visually checked that it’s JSON compliant, and then started modifying the complex fields one by one to determine which field is causing the import to fail. This lead to the “to” field.

Afterwards, I checked whether mongoimport 3.0 can deal with the syntax that mongoimport 2.6 wasn’t able to, and then did a cross check to ensure that this is indeed a mongoimport issue, and not something related to the MongoDB server version 2.6.