- Administration >
- Administration Concepts >
- Data Management >
- Capped Collections
Capped Collections¶
On this page
Overview¶
Capped collections are fixed-size collections that support high-throughput operations that insert and retrieve documents based on insertion order. Capped collections work in a way similar to circular buffers: once a collection fills its allocated space, it makes room for new documents by overwriting the oldest documents in the collection.
See createCollection() or create for more information on creating capped collections.
Behavior¶
Insertion Order¶
Capped collections guarantee preservation of the insertion order. As a result, queries do not need an index to return documents in insertion order. Without this indexing overhead, capped collections can support higher insertion throughput.
Automatic Removal of Oldest Documents¶
To make room for new documents, capped collections automatically remove the oldest documents in the collection without requiring scripts or explicit remove operations.
For example, the oplog.rs collection that stores a log of the operations in a replica set uses a capped collection. Consider the following potential use cases for capped collections:
- Store log information generated by high-volume systems. Inserting documents in a capped collection without an index is close to the speed of writing log information directly to a file system. Furthermore, the built-in first-in-first-out property maintains the order of events, while managing storage use.
- Cache small amounts of data in a capped collections. Since caches are read rather than write heavy, you would either need to ensure that this collection always remains in the working set (i.e. in RAM) or accept some write penalty for the required index or indexes.
_id Index¶
Changed in version 2.4.
Capped collections have an _id field and an index on the _id field by default.
Restrictions and Recommendations¶
Updates¶
If you plan to update documents in a capped collection, create an index so that these update operations do not require a table scan.
With MMAPv1, you can only make in-place updates of documents. If the update operation causes a document to grow beyond the document’s original size, the update operation will fail.
Replica Sets with MMAPv1 Secondaries¶
If you update a document in a capped collection to a size smaller than its original size and a secondary resyncs from the primary, the secondary will replicate and allocate space based on the current smaller document size.
If the primary then receives an update which increases the document back to its original size, the primary will accept the update. However, for MMAPv1, the secondary will fail with a failing update: objects in a capped ns cannot grow error message.
To prevent this error, create your secondary from a snapshot of one of the other up-to-date members of the replica set. Follow the :doc:` tutorial on filesystem snapshots </tutorial/backup-with-filesystem-snapshots>` to seed your new secondary.
Seeding the secondary with a filesystem snapshot is the only way to guarantee the primary and secondary binary files are compatible. MongoDB Cloud Manager Backup snapshots are insufficient in this situation since you need more than the content of the secondary to match the primary.
Document Deletion¶
You cannot delete documents from a capped collection. To remove all documents from a collection, use the drop() method to drop the collection and recreate the capped collection.
Sharding¶
You cannot shard a capped collection.
Query Efficiency¶
Use natural ordering to retrieve the most recently inserted elements from the collection efficiently. This is (somewhat) analogous to tail on a log file.
Procedures¶
Create a Capped Collection¶
You must create capped collections explicitly using the createCollection() method, which is a helper in the mongo shell for the create command. When creating a capped collection you must specify the maximum size of the collection in bytes, which MongoDB will pre-allocate for the collection. The size of the capped collection includes a small amount of space for internal overhead.
db.createCollection( "log", { capped: true, size: 100000 } )
If the size field is less than or equal to 4096, then the collection will have a cap of 4096 bytes. Otherwise, MongoDB will raise the provided size to make it an integer multiple of 256.
Additionally, you may also specify a maximum number of documents for the collection using the max field as in the following document:
db.createCollection("log", { capped : true, size : 5242880, max : 5000 } )
Important
The size argument is always required, even when you specify max number of documents. MongoDB will remove older documents if a collection reaches the maximum size limit before it reaches the maximum document count.
See
createCollection() and create.
Query a Capped Collection¶
If you perform a find() on a capped collection with no ordering specified, MongoDB guarantees that the ordering of results is the same as the insertion order.
To retrieve documents in reverse insertion order, issue find() along with the sort() method with the $natural parameter set to -1, as shown in the following example:
db.cappedCollection.find().sort( { $natural: -1 } )
Check if a Collection is Capped¶
Use the isCapped() method to determine if a collection is capped, as follows:
db.collection.isCapped()
Convert a Collection to Capped¶
You can convert a non-capped collection to a capped collection with the convertToCapped command:
db.runCommand({"convertToCapped": "mycoll", size: 100000});
The size parameter specifies the size of the capped collection in bytes.
Warning
This command obtains a global write lock and will block other operations until it has completed.
Changed in version 2.2: Before 2.2, capped collections did not have an index on _id unless you specified autoIndexId to the create, after 2.2 this became the default.
Automatically Remove Data After a Specified Period of Time¶
For additional flexibility when expiring data, consider MongoDB’s TTL indexes, as described in Expire Data from Collections by Setting TTL. These indexes allow you to expire and remove data from normal collections using a special type, based on the value of a date-typed field and a TTL value for the index.
TTL Collections are not compatible with capped collections.
Tailable Cursor¶
You can use a tailable cursor with capped collections. Similar to the Unix tail -f command, the tailable cursor “tails” the end of a capped collection. As new documents are inserted into the capped collection, you can use the tailable cursor to continue retrieving documents.
See Create Tailable Cursor for information on creating a tailable cursor.
Thank you for your feedback!
We're sorry! You can Report a Problem to help us improve this page.