Hadoop Streaming with Python
Hadoop Streaming Hadoop streaming is a utility that comes with the Hadoop distribution. The utility allows developers to create an run Map/Reduce jobs with any executable or script as the ampper and/or the reducer. For example: hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/contrib/streaming/hadoop-streaming-2.0.0-mr1-cdh4.5.0.jar \…
MongoDB Java driver testing
1. Install latest JDK(jdk-8u5-linux-x64.rpm) 2. Download MongoDB Java driver(mongo-java-driver-2.12.1.jar) 3. Export the variables / run the program in the below given format by keeping jar file in the same folder. 3. Run the java code example code;- [root@myhostname JAVA]$ cat…
MongoDB testing Java Driver with SSL & Kerberos Enabled
[root@myhostname srini]# cat SSLKerb.java import com.mongodb.*; import javax.net.ssl.SSLSocketFactory; import java.net.UnknownHostException; import java.security.Security; import static java.util.Arrays.asList; public class SSLKerb { public static void main(String[] args) throws UnknownHostException, InterruptedException { System.setProperty(“javax.net.ssl.trustStore”,”/usr/java/jdk1.8.0_05/jre/lib/security/cacerts”); System.setProperty(“javax.net.ssl.trustStorePassword”,”changeit”); System.setProperty(“javax.net.ssl.trustStoreType”,”jks”); System.setProperty(“javax.security.auth.useSubjectCredsOnly”, “false”); System.setProperty(“java.security.krb5.realm”, “ASIA.NSTSRIN.NET”); System.setProperty(“java.security.krb5.kdc”, “myhostname.ASIA.NSTSRIN.NET”); String user =…
[MongoDB]: restore vs import
Utility mongorestore allows you to load data into the database. It requires a file (dump) that contains a BSON . Such a file can be created using different tools mongodump . The second utility mongoimport allows you to import data…
History of MySQL
Inception MySQL was created by a Swedish company MySQL AB in 1995. The developers of the platform were Michael Widenius (Monty), David Axmark and Allan Larsson. The foremost purpose was to provide efficient and reliable data management options to…
mtool for MongoDB Diagnostics
mtools mtools is a collection of helper scripts to parse and filter MongoDB log files (mongod, mongos), visualize log files and quickly set up complex MongoDB test environments on a local machine. Installation procedure :- Step 1:…
MongoDB as pure In-memory
MongoDB as an pure in-memory database, meaning that the data is not stored on disk at all. This can be super useful for applications like: A write-heavy cache in front of a slower RDBMS system Embedded systems PCI compliant systems…
mdiag script for gathering MongoDB’s system & h/w diagnostic info.
mdiag is a shell script which will gather a wide variety of system and hardware diagnostic information of the MongoDB server. Please see below for how it works. mdiag shell script: [Lab root @ hostname /tmp]# cat mdiag.sh #!/bin/sh…
Importing large flat files into mongoDB
This is a very basic technique, but that’s how I like to start. I will also show a couple tricks when working with large data files. Editing large files Let’s assume you have a large data file, approximately 60MB with…
Monitoring Hadoop from the browser
Hadoop provides two web interfaces that you should become familiar with, one for HDFS and the other for MapReduce. Both are useful in pseudo-distributed mode and are critical tools when you have a fully distributed setup. The HDFS web UI…