MongoDB: Understanding Oplog

The following table clearly explains what each Oplog field denotes.

 

Field Name	Alias	Details	Operations	Values
ts	TimeSpace	Holds the “Time of the operation” & “incremental id”. For all bulk (insert/remove/update) operations “Time of the operation” remains same and id changes to an incremental value. Default value of the id is 1	For All Operations	(Time of the operation, incremental id)
h	hash	a unique ID for each operation	For All Operations	NumberLong(“7202552052460497158”)
t	NA	NA	For All Operations	NumberLong(1)
v	version#	version	For All Operations	2
op	Operation Type	Write operations (import/Insert/Delete/Update/)	Write Operations	i/d/u
		Commands that affect database (Create, Drop: Database/collection)	Command Operations	C
		Informational Operation	No-Operation	N
ns	NameSpace	The database and collection affected by the operation	Write Operations	(db_name.collection_name)
			Command Operations	(db_name.$cmd))
			No-Operation	Blank
O	Operation	Info/Data of the Operations	Initiate Replica Set	{“msg” : “initiate set”}
			On Electing as Primary	{“msg” : “new primary”}
			Add Node1 as Secondary	{“msg” : “Reconfig Set”, “version”:2}
			Add Node2 as Secondary	{“msg” : “Reconfig Set”, “version”:3}
			Remove Node1 from RepSet	{“msg” : “Reconfig Set”, “version”:4}
			Create Collection	{ “create” : “posts” }
			Drop Collection	{ “drop” : “posts” }
			Drop Database	{ “dropDatabase” : 1 }
			Insert/Delete a Document	{“_id”:1, “Name”:”ABC”}
			Set () for update	{ “$set” : { “Name” : “MyTest” } }
O2	Update Operation	Holds the _id of the document to be updated	Update Document	{ “_id” : 21 }

 

 

 

Oplog Default Size

When you start a replica set member for the first time, MongoDB creates an oplog of a default size.

 

Default oplog size in Windows/Unix   :

5% of physical memory/free disk space

50MB to 50GB of physical memory

990MB to 50GB of free disk space

 

Default oplog size in OS X 64 bit        :

192MB of physical memory/free disk space

 

Change the size of Oplog

 

– The procedure to be carried on maintenance period

– Always start with the Secondary nodes first and lastly on primary

– One node at a time

– Restart node in “Standalone” mode on a different port

 

Let’s assume this is 3 node replication set:

Node	Port	Replica Set	DB Path	Log Path
Node1	27011	Primary	C:\db\node1	C:\db\node1\log\node1.log
Node2	27012	Secondary	C:\db\node2	C:\db\node2\log\node2.log
Node3	27013	Secondary	C:\db\node3	C:\db\node3\log\node3.log

 

*Note: I’ve used variables %node%  %Port% %DBpath% and %LogPath% to simplify the instructions. Wherever you see these variables replace them with the corresponding values from the above table for practice

 

First Start with the secondary nodes one at a time

For  Secondary Nodes 2 & 3:

Step1: Check the present Oplog size

rs.printReplicationInfo()

 

Step2: Shutdown the node

use admin

db.shutdownServer()

 

Step3: Restart mongod daemon as standalone on differ port 27014

mongod –dbpath %dbpath% –logpath %logpath% –port 27014

 

Step4: Backup existing Oplog using mongodump

mongodump –db local –collection oplog.rs –port 27014 –out “C:\oplog\ %node%”

 

Step5: Start the client

Mongo –port 27014

 

Step6: Switch to local database

use local

 

Step7: drop collection “temp” (if exists) and then create temp collection

db.temp.drop()

db.createCollection(“temp”)

 

Step7: Insert the last Oplog entry into temp collection

db.temp.save(db.oplog.rs.find({},{ts:1, h:1}).sort({$natural: -1}).limit(1).next()))

 

Step8: check the temp collection

db.temp.find()

 

Step9: Delete the present oplog.rs collection

db.oplog.rs.drop()

 

Step10: Create a new capped collection with name “oplog.rs”

 

 

db.runCommand( { create:”oplog.rs”, capped: true, size: (2*1024*1024*1024) } )

// 2*1024*1024*1024 equals 2GB

 

Step11: Insert the old Oplog last entry into new Oplog collection

db.oplog.rs.save(db.temp.findOne())

 

Step12: Check the new Oplog collection

db.oplog.rs.find()

 

Step13: Shutdown the client

use admin

db.shutdownServer()

 

Step14: Exit

Exit

 

Step15: Restart mongod daemon as member on original port

mongod –replSet rs  –dbpath %dbpath% –logpath %logpath% –smallfiles –nojournal –storageEngine=mmapv1  –port %port%

 

Step16: Restart mongo client on original port %port%

mongo –port %port%

 

Step17: New Oplog Size

rs.printReplicationInfo()

 

For Primary Node1:

 

Step A: Step down the node to secondary

rs.stepDown()

 

Step B: Repeat the above procedure from Step1 to Step17

 

 

 

Ask Question