[MongoDB]: how to get better performance for mongoimport operation
1) Check if your db/collection have many indexes? If so we could test if reducing the number of indexes is an option, based on your reads and query patterns.
2) Since prior to MongoDB 3.0, locking is at Database level and hence you may experience slowness. Locking is at document level from MongoDB 3.0 onwards.
http://docs.mongodb.org/manual/release-notes/3.0/
You can also upgrade the mongodb-org-tools to 3.0.3 package, not the mongodb-org-server (or mongod process).
As per the MongoDB Inc. there is lot of performance improvement with 3.x version 7 it’s tools.
Refer to the link
mongodb-org-tools=3.0.3
http://docs.mongodb.org/v3.0/administration/install-on-linux/
3) Also you can split load to multiple databases.
Imagine if you’ve 10 different collections data in your CSV : This approach would require you have 10 databases, use mongoimport 10 times and then move the documents or collections into a single database.
But this approach is likely to be less efficient and more prone to errors.
Also, you can test running 10 mongoimport parallel processes inserting csv data from a file that was previously split into 10 parts as well, but always import into the same database and collection.
4) Another tool you can use monitor the speed of inserts is mongostat.
If you run this process in another terminal window, you’ll be able to see the number of operations per second and other performance-related metrics.
5) Please check the disk utilization (where the dbfiles are stored ), it shouldn’t be bottleneck.