[MongoDB]: Tailable cursors
Tailable cursors are cursors that remain open even after they’ve returned a final result. This way, if more documents are added to a collection (i.e., to the cursor’s result set), then you can continue to call Cursor#next to retrieve those results.
By default, MongoDB will automatically close a cursor when the client has exhausted all results in the cursor. However, for capped collections you may use a Tailable Cursor that remains open after the client exhausts the results in the initial cursor. Tailable cursors are conceptually equivalent to the tail Unix command with the -f option (i.e. with “follow” mode). After clients insert new additional documents into a capped collection, the tailable cursor will continue to retrieve documents.
Use tailable cursors on capped collections that have high write volumes where indexes aren’t practical. For instance, MongoDB replication uses tailable cursors to tail the primary’s oplog.
NOTE
If your query is on an indexed field, do not use tailable cursors, but instead, use a regular cursor. Keep track of the last value of the indexed field returned by the query. To retrieve the newly added documents, query the collection again using the last value of the indexed field in the query criteria, as in the following example:
db.<collection>.find( { indexedField: { $gt: <lastvalue> } } )
Consider the following behaviors related to tailable cursors:
Tailable cursors do not use indexes and return documents in natural order.
Because tailable cursors do not use indexes, the initial scan for the query may be expensive; but, after initially exhausting the cursor, subsequent retrievals of the newly added documents are inexpensive.
Tailable cursors may become dead, or invalid, if either:
the query returns no match.
the cursor returns the document at the “end” of the collection and then the application deletes that document.
A dead cursor has an id of 0.
Let’s test cursors as below.
> show dbs
admin (empty)
local 0.031GB
test (empty)
>
>
>
>
> use dbversity
switched to db dbversity
>
> db.createCollection(‘messages’, {capped: true, size: 1000000})
{ “ok” : 1 }
>
>
> show collections
messages
system.indexes
>
>
Inserting bulk data through a for loop at another promt :
——————————————————-
> for(var i = 1; i <= 10000 ; i++){db.messages.insert({“id” : i , “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : i }); sleep(30);}
—————————————–
> db.messages.isCapped()
true
>
>
> db.messages.find()
{ “_id” : ObjectId(“552511da77a94260b30bea8f”), “id” : 1, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 1 }
{ “_id” : ObjectId(“552511da77a94260b30bea90”), “id” : 2, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 2 }
{ “_id” : ObjectId(“552511da77a94260b30bea91”), “id” : 3, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 3 }
{ “_id” : ObjectId(“552511da77a94260b30bea92”), “id” : 4, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 4 }
{ “_id” : ObjectId(“552511da77a94260b30bea93”), “id” : 5, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 5 }
{ “_id” : ObjectId(“552511da77a94260b30bea94”), “id” : 6, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 6 }
{ “_id” : ObjectId(“552511da77a94260b30bea95”), “id” : 7, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 7 }
{ “_id” : ObjectId(“552511da77a94260b30bea96”), “id” : 8, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 8 }
{ “_id” : ObjectId(“552511da77a94260b30bea97”), “id” : 9, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 9 }
{ “_id” : ObjectId(“552511da77a94260b30bea98”), “id” : 10, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 10 }
{ “_id” : ObjectId(“552511da77a94260b30bea99”), “id” : 11, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 11 }
{ “_id” : ObjectId(“552511da77a94260b30bea9a”), “id” : 12, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 12 }
{ “_id” : ObjectId(“552511da77a94260b30bea9b”), “id” : 13, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 13 }
{ “_id” : ObjectId(“552511da77a94260b30bea9c”), “id” : 14, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 14 }
{ “_id” : ObjectId(“552511da77a94260b30bea9d”), “id” : 15, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 15 }
{ “_id” : ObjectId(“552511da77a94260b30bea9e”), “id” : 16, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 16 }
{ “_id” : ObjectId(“552511db77a94260b30bea9f”), “id” : 17, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 17 }
{ “_id” : ObjectId(“552511db77a94260b30beaa0”), “id” : 18, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 18 }
{ “_id” : ObjectId(“552511db77a94260b30beaa1”), “id” : 19, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 19 }
{ “_id” : ObjectId(“552511db77a94260b30beaa2”), “id” : 20, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 20 }
Type “it” for more
>
>
> var t = db.messages
> var cursor = t.find().addOption(DBQuery.Option.tailable).addOption(DBQuery.Option.awaitData)
>
> cursor
{ “_id” : ObjectId(“552511da77a94260b30bea8f”), “id” : 1, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 1 }
{ “_id” : ObjectId(“552511da77a94260b30bea90”), “id” : 2, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 2 }
{ “_id” : ObjectId(“552511da77a94260b30bea91”), “id” : 3, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 3 }
{ “_id” : ObjectId(“552511da77a94260b30bea92”), “id” : 4, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 4 }
{ “_id” : ObjectId(“552511da77a94260b30bea93”), “id” : 5, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 5 }
{ “_id” : ObjectId(“552511da77a94260b30bea94”), “id” : 6, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 6 }
{ “_id” : ObjectId(“552511da77a94260b30bea95”), “id” : 7, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 7 }
{ “_id” : ObjectId(“552511da77a94260b30bea96”), “id” : 8, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 8 }
{ “_id” : ObjectId(“552511da77a94260b30bea97”), “id” : 9, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 9 }
{ “_id” : ObjectId(“552511da77a94260b30bea98”), “id” : 10, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 10 }
{ “_id” : ObjectId(“552511da77a94260b30bea99”), “id” : 11, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 11 }
{ “_id” : ObjectId(“552511da77a94260b30bea9a”), “id” : 12, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 12 }
{ “_id” : ObjectId(“552511da77a94260b30bea9b”), “id” : 13, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 13 }
{ “_id” : ObjectId(“552511da77a94260b30bea9c”), “id” : 14, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 14 }
{ “_id” : ObjectId(“552511da77a94260b30bea9d”), “id” : 15, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 15 }
{ “_id” : ObjectId(“552511da77a94260b30bea9e”), “id” : 16, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 16 }
{ “_id” : ObjectId(“552511db77a94260b30bea9f”), “id” : 17, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 17 }
{ “_id” : ObjectId(“552511db77a94260b30beaa0”), “id” : 18, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 18 }
{ “_id” : ObjectId(“552511db77a94260b30beaa1”), “id” : 19, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 19 }
{ “_id” : ObjectId(“552511db77a94260b30beaa2”), “id” : 20, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 20 }
Type “it” for more
>
>
> cursor
{ “_id” : ObjectId(“552511db77a94260b30beaa3”), “id” : 21, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 21 }
{ “_id” : ObjectId(“552511db77a94260b30beaa4”), “id” : 22, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 22 }
{ “_id” : ObjectId(“552511db77a94260b30beaa5”), “id” : 23, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 23 }
{ “_id” : ObjectId(“552511db77a94260b30beaa6”), “id” : 24, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 24 }
{ “_id” : ObjectId(“552511db77a94260b30beaa7”), “id” : 25, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 25 }
{ “_id” : ObjectId(“552511db77a94260b30beaa8”), “id” : 26, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 26 }
{ “_id” : ObjectId(“552511db77a94260b30beaa9”), “id” : 27, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 27 }
{ “_id” : ObjectId(“552511db77a94260b30beaaa”), “id” : 28, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 28 }
{ “_id” : ObjectId(“552511db77a94260b30beaab”), “id” : 29, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 29 }
{ “_id” : ObjectId(“552511db77a94260b30beaac”), “id” : 30, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 30 }
{ “_id” : ObjectId(“552511db77a94260b30beaad”), “id” : 31, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 31 }
{ “_id” : ObjectId(“552511db77a94260b30beaae”), “id” : 32, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 32 }
{ “_id” : ObjectId(“552511db77a94260b30beaaf”), “id” : 33, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 33 }
{ “_id” : ObjectId(“552511db77a94260b30beab0”), “id” : 34, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 34 }
{ “_id” : ObjectId(“552511db77a94260b30beab1”), “id” : 35, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 35 }
{ “_id” : ObjectId(“552511db77a94260b30beab2”), “id” : 36, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 36 }
{ “_id” : ObjectId(“552511db77a94260b30beab3”), “id” : 37, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 37 }
{ “_id” : ObjectId(“552511db77a94260b30beab4”), “id” : 38, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 38 }
{ “_id” : ObjectId(“552511db77a94260b30beab5”), “id” : 39, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 39 }
{ “_id” : ObjectId(“552511db77a94260b30beab6”), “id” : 40, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 40 }
Type “it” for more
>
>
> db.messages.count()
2448
>
> cursor
{ “_id” : ObjectId(“552511db77a94260b30beab7”), “id” : 41, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 41 }
{ “_id” : ObjectId(“552511db77a94260b30beab8”), “id” : 42, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 42 }
{ “_id” : ObjectId(“552511db77a94260b30beab9”), “id” : 43, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 43 }
{ “_id” : ObjectId(“552511db77a94260b30beaba”), “id” : 44, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 44 }
{ “_id” : ObjectId(“552511db77a94260b30beabb”), “id” : 45, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 45 }
{ “_id” : ObjectId(“552511db77a94260b30beabc”), “id” : 46, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 46 }
{ “_id” : ObjectId(“552511db77a94260b30beabd”), “id” : 47, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 47 }
{ “_id” : ObjectId(“552511dc77a94260b30beabe”), “id” : 48, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 48 }
{ “_id” : ObjectId(“552511dc77a94260b30beabf”), “id” : 49, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 49 }
{ “_id” : ObjectId(“552511dc77a94260b30beac0”), “id” : 50, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 50 }
{ “_id” : ObjectId(“552511dc77a94260b30beac1”), “id” : 51, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 51 }
{ “_id” : ObjectId(“552511dc77a94260b30beac2”), “id” : 52, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 52 }
{ “_id” : ObjectId(“552511dc77a94260b30beac3”), “id” : 53, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 53 }
{ “_id” : ObjectId(“552511dc77a94260b30beac4”), “id” : 54, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 54 }
{ “_id” : ObjectId(“552511dc77a94260b30beac5”), “id” : 55, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 55 }
{ “_id” : ObjectId(“552511dc77a94260b30beac6”), “id” : 56, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 56 }
{ “_id” : ObjectId(“552511dc77a94260b30beac7”), “id” : 57, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 57 }
{ “_id” : ObjectId(“552511dc77a94260b30beac8”), “id” : 58, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 58 }
{ “_id” : ObjectId(“552511dc77a94260b30beac9”), “id” : 59, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 59 }
{ “_id” : ObjectId(“552511dc77a94260b30beaca”), “id” : 60, “info” : “How do I create manual workload i.e., Bulk inserts to Collection “, ” Iteration no:” : 60 }
Type “it” for more
>
>