Install Hadoop CDH 4/5, Hive, Pig, Java & MongoDB and set environment variables as below. [root@dbversity.com ~]# cat ~/.bashrc | grep “export” export PATH=$PATH:/opt/mongodb/bin export JAVA_HOME=/usr/java/jdk1.8.0_05 export PATH=$JAVA_HOME/bin:$PATH export HADOOP_HOME=/hadoop export PATH=$HADOOP_HOME/bin:$PATH export HIVE_HOME=/hadoop/hive-0.12.0-cdh5.0.0 export PATH=$HIVE_HOME/bin:$PATH export PIG_HOME=/hadoop/pig-0.12.0-cdh5.0.0 export PATH=$PIG_HOME/bin:$PATH…
[root @ dbversity /tmp]# java -version java version “1.8.0_05” Java(TM) SE Runtime Environment (build 1.8.0_05-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.5-b02, mixed mode) [root @ dbversity /tmp] [root @ dbversity /tmp]# which java /usr/java/jdk1.8.0_05/bin/java Otherway to know the installed…
What is the significance of _tmp directory is used for in MongoDB and why it gets created? There are at least two ways from MongoDB to create a _tmp directory. INITIAL FILE ALLOCATION : – ======================== During the file allocation,…
> use dbversitydb switched to db dbversitydb > > > for(i = 1; i <= 1000; i++) db.dbversity_website.insert( { post_id : i, comment_id : i, likes : i}); WriteResult({ “nInserted” : 1 }) > > > > db.dbversity_website.findOne() { “_id”…
MongoDB dropDups option in Index creation will be useful to remove duplicate records that already exist in the collection. > use foo switched to db foo > > > > db.test.insert({ a:1 , b:1, c:1 }) WriteResult({ “nInserted” : 1…
MongoDB allows you to specify a unique constraint on an index. These constraints prevent applications from inserting documents that have duplicate values for the inserted fields. > db.dbversity.createIndex( { “dbversity.post_id” : 1 } , { unique : true } )…
TTL (Time To Live) indexes are special single-field indexes that MongoDB can use to automatically remove documents from a collection after a certain amount of time. Data expiration is useful for certain types of information like machine generated event data,…
DBRefs vs Manual References As an example scenario where we would use DBRefs instead of Manual References, consider a database where we are storing different types of addresses (home, office, mailing, etc) in different collections (address_home, address_office, address_mailing, etc). Now,…
MongoDB Fragmentation Alerting script :- 1) Script will consider below aspects and prompt us “POSSIBLE FRAGMENTATION” & “REVIEW SCHEMA” issues. dataSize indexSize storageSize fileSize 2) It has 6GB minimum threshold dataSize for the database as thumb rule to apply the…