Importing json from file into mongodb using mongoimport
MongodbMongodb Problem Overview
I have my json_file.json like this:
[{ "project": "project_1", "coord1": 2, "coord2": 10, "status": "yes", "priority": 7},{ "project": "project_2", "coord1": 2, "coord2": 10, "status": "yes", "priority": 7},{ "project": "project_3", "coord1": 2, "coord2": 10, "status": "yes", "priority": 7}]
When I run the following command to import this into mongodb:
mongoimport --db my_db --collection my_collection --file json_file.json
I get the following error:
Failed: error unmarshaling bytes on document #0: JSON decoder out of sync - data changing underfoot?
If I add the --jsonArray flag to the command I import like this:
imported 3 documents
instead of one document with the json format as shown in the original file.
How can I import json into mongodb with the original format in the file shown above?
Mongodb Solutions
Solution 1 - Mongodb
The mongoimport
tool has an option:
>--jsonArray
treat input source as a JSON array
Or it is possible to import from file containing same data format as the result of db.collection.find()
command. Here is example from university.mongodb.com courseware some content from grades.json
:
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb577" }, "student_id" : 0, "type" : "exam", "score" : 54.6535436362647 }
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb578" }, "student_id" : 0, "type" : "quiz", "score" : 31.95004496742112 }
{ "_id" : { "$oid" : "50906d7fa3c412bb040eb579" }, "student_id" : 0, "type" : "homework", "score" : 14.8504576811645 }
As you can see, no array used and no comma delimiters between documents either.
I discover, recently, that this complies with the JSON Lines text
format.
Like one used in apache.spark.sql.DataFrameReader.json()
method.
Solution 2 - Mongodb
Perhaps the following reference from the MongoDB project blog could help you gain insight on how arrays work in Mongo:
https://blog.mlab.com/2013/04/thinking-about-arrays-in-mongodb/
I would frame your import otherwise, and either:
a) import the three different objects separately into the collection as you say, using the --jsonArray flag; or
b) encapsulate the complete array within a single object, for example in this way:
{
"mydata":
[
{
"project": "project_1",
...
"priority": 7
}
]
}
HTH.
Solution 3 - Mongodb
I faced opposite problem today, my conclusion would be:
If you wish to insert array of JSON objects at once, where each array entry shall be treated as separate dtabase entry, you have two options of syntax:
-
Array of object with valid coma positions & --jsonArray flag obligatory
[ {obj1}, {obj2}, {obj3} ]
-
Use file with basically incorrect JSON formatting (i.e. missing
,
between JSON object instances & without --jsonArray flag{obj1} {obj2} {obj3}
If you wish to insert only an array (i.e. array as top-level citizen of your database) I think it's not possible and not valid, because mongoDB by definition supports documents as top-level objects which are mapped to JSON objects afterwards. In other words, you must wrap your array into JSON object as ALAN WARD pointed out.
Solution 4 - Mongodb
Error:
$ ./mongoimport --db bookings --collection user --file user.json
2021-06-12T18:52:13.256+0530 connected to: localhost
2021-06-12T18:52:13.261+0530 Failed: error unmarshaling bytes on document #0: JSON decoder out of sync - data changing underfoot?
2021-06-12T18:52:13.261+0530 imported 0 documents
Solution: When your JSON data contain an array of objects then we need to use --jsonArray while import with the command like mentioned below
$ ./mongoimport --db bookings --collection user --file user.json --jsonArray
2021-06-12T18:53:44.164+0530 connected to: localhost
2021-06-12T18:53:44.532+0530 imported 414 documents