MongoDB Shrink database for cleanup space released
When documents or collections are deleted, empty record blocks within data files arise. MongoDB attempts to reuse this space when possible, but it will never return this space to the file system. This behavior explains why fileSize never decreases despite…
Aggregation Framework Operators
Pipeline Operators :- NOTE The aggregation pipeline cannot operate on values of the following types: Symbol, MinKey, MaxKey,DBRef, Code, and CodeWScope. Pipeline operators appear in an array. Documents pass through the operators in a sequence. Name Description $project Reshapes a…
Administering Hadoop
Namenode directory structure :- —————————————— A newly formatted namenode creates the following directory structure: ${dfs.name.dir}/current/VERSION /edits /fsimage /fstime In my machine [root@myhostname current]# pwd /data/2/hadoop/tmp/dfs/name/current [root@myhostname current]# [root@myhostname current]# ll -lhtr total 16K -rw-r–r– 1 root root 110 Jul 22…
MongoDB High Availability Min configuration
If you have 1 Primary node, 1 Secondary node and 1 Arbiter, and you are using majority write, then you are requiring that the write be made on both the Primary and the Secondary. As soon as one of those…
MongoDB Aggregation Concepts
MongoDB provides the three approaches to aggregation, each with its own strengths and purposes for a given situation. This section describes these approaches and also describes behaviors and limitations specific to each approach. See also the chart that compares the…
MapReduce Job [hadoop]
Running our first MapReduce job We will use the WordCount example job which reads text files and counts how often words occur. The input is text files and the output is text files, each line of which contains a word…
password option file in MongoDB
if you’re curious to know whether we’ve password option file concept in MongoDB something similar to MySQL (refer below links). http://dev.mysql.com/doc/refman/5.1/en/password-security-user.html http://stackoverflow.com/questions/16299603/mysql-utilities-my-cnf-option-file With this, we can actually save credentials in a option hidden file within client session and we no…
NUMA (Non Uniform Memory Access) for MongoDB
We usually did see some performance boost when NUMA was disabled using -> numactl –interleave=all ….. Below are the details from MongoDB http://docs.mongodb.org/manual/administration/production-notes/ Running MongoDB on a system with Non-Uniform Access Memory (NUMA) can cause a number of operational problems,…
Getting started with Hive
Hive is a data warehouse that uses MapReduce to analyze data stored on HDFS. In particular, it provides a query language called HiveQL that closely resembles the common Structured Query Language (SQL) standard. Prerequisites Unlike Hadoop, there are no Hive…
MongoDB Queries
–> To find documents in a collection we have syntax db.<collection Name>.find(); –> db.books.find(); — it will provide all documents in collection books. It actually returns a cursor. –> To make the output look a bit pretty we have following…