Understanding MongoDB Journal, Index Data and Diagnostics Files …

February 22, 2019 · by · in MongoDB

There is no available tool to read/view the index files or journal files for a mongod instance.

You can inspect the diagnostic.data/metrics.* files using the bsondump utility. The contents are outputs of the db.serverStatus() command saved in a binary format.

The index files and journal files should not be viewed directly. Instead, it is recommended and best practice to view information relating to indexes, by running commands from the mongo shell to query the database. For example, to get a list of indexes for the test collection, run the following command:

db.test.getIndexes()

The index files, journal files, WiredTiger files are binary files stored in binary format. These are only readable by the MongoDB database and is not human readable, thus it means we cannot sample/view these binary files directly.Also, let us know the signification of all .wt, .turtle, .bson files in the dbpath (data directory) and let us know how to read them.

When using MongoDB with the WiredTiger storage engine, there are different sets of files in the $dbpath.

MongoDB internal and lock files:
storage.bson
mongod.lock

WiredTiger metadata files:
WiredTiger.lock
WiredTiger
WiredTigerLAS.wt
sizeStorer.wt
_mdb_catalog.wt
WiredTiger.wt
WiredTiger.turtle

Descriptions of each of the files and its functions are as follows:

  • WiredTiger: the WiredTiger version file

This text file contains the version of WiredTiger used to create the database, and is used by WiredTiger on startup to determine whether a database is present.

  • WiredTiger.wt: the WiredTiger metadata file

The main metadata file that tracks information about which collections and indexes exist in the database, as well as the most recent durable update (checkpoint) for each of them.

  • WiredTiger.turtle: engine configuration file for the WiredTiger.wt file

This text file contains the WiredTiger engine configuration for the WiredTiger.wt metadata file described above.

  • WiredTigerLAS.wt: the WiredTiger lookaside file

MongoDB configures WiredTiger in-memory cache to contain no more than a specified percentage of dirty data. If the dirty data exceeds the configured number and WiredTiger is required to keep the dirty data for operational reasons, this file contains the disk spill of the dirty data.

  • _mdb_catalog.wt: the MongoDB metadata file

This file contains the metadata and catalog of MongoDB databases, collections, and indexes, including which WiredTiger data file maps to each MongoDB collection or index.

  • WiredTiger.lock and mongod.lock: lock files

MongoDB and WiredTiger use these lock files to ensure that only a single process accesses the database at any particular time.

  • sizeStorer.wt: the MongoDB collection size tracking file

This file contains information about the size and document count of each MongoDB collection.

  • collection-X-XXX.wt and index-X-XXX.wt: MongoDB collection and index files

Each MongoDB collection is stored in a collection-X-XXX.wt file, and each MongoDB index is stored in a index-X-XXX.wt file. The X-XXXportion of the filename represents a random number used by WiredTiger as a unique internal identifier. If MongoDB is started with the --directoryPerDB option these files may appear in a subdirectory of the dbpath.

Files created by optional settings

  • journal directory: WiredTiger journal files

If a database is started without the --nojournal option, the database directory also contain a journal subdirectory that has the following contents:

WiredTigerLog.0000000001
WiredTigerPreplog.0000000001
WiredTigerPreplog.0000000002

Of importance here is the WiredTigerLog.XX file, which contains the Write Ahead Log (WAL) maintained by WiredTiger.

A note of caution, WiredTiger files and directories should not be manipulated outside of MongoDB.

Do not remove or modify any of these files. Doing so may leave the database in an inoperable state.

  • Ask Question
    • Tags: ,