Navigation

Monitoring for MongoDB

On this page

Monitoring is a critical component of all database administration. A firm grasp of MongoDB’s reporting will allow you to assess the state of your database and maintain your deployment without crisis. Additionally, a sense of MongoDB’s normal operational parameters will allow you to diagnose problems before they escalate to failures.

This document presents an overview of the available monitoring utilities and the reporting statistics available in MongoDB. It also introduces diagnostic strategies and suggestions for monitoring replica sets and sharded clusters.

Note

MongoDB Atlas is a cloud-hosted database-as-a-service. MongoDB Cloud Manager, a hosted service, and Ops Manager, an on-premise solution, provide monitoring, backup, and automation of MongoDB instances. For documentation, see Atlas documentation, the MongoDB Cloud Manager documentation and Ops Manager documentation

Monitoring Strategies

There are three methods for collecting data about the state of a running MongoDB instance:

Each strategy can help answer different questions and is useful in different contexts. These methods are complementary.

MongoDB Reporting Tools

This section provides an overview of the reporting methods distributed with MongoDB. It also offers examples of the kinds of questions that each method is best suited to help you address.

Utilities

The MongoDB distribution includes a number of utilities that quickly return statistics about instances’ performance and activity. Typically, these are most useful for diagnosing issues and assessing normal operation.

mongostat

mongostat captures and returns the counts of database operations by type (e.g. insert, query, update, delete, etc.). These counts report on the load distribution on the server.

Use mongostat to understand the distribution of operation types and to inform capacity planning. See the mongostat manual for details.

mongotop

mongotop tracks and reports the current read and write activity of a MongoDB instance, and reports these statistics on a per collection basis.

Use mongotop to check if your database activity and use match your expectations. See the mongotop manual for details.

HTTP Console

Deprecated since version 3.2: HTTP interface for MongoDB

MongoDB provides a web interface that exposes diagnostic and monitoring information in a simple web page. The web interface is accessible at localhost:<port>, where the <port> number is 1000 more than the mongod port .

For example, if a locally running mongod is using the default port 27017, access the HTTP console at http://localhost:28017.

Commands

MongoDB includes a number of commands that report on the state of the database.

These data may provide a finer level of granularity than the utilities discussed above. Consider using their output in scripts and programs to develop custom alerts, or to modify the behavior of your application in response to the activity of your instance. The db.currentOp method is another useful tool for identifying the database instance’s in-progress operations.

serverStatus

The serverStatus command, or db.serverStatus() from the shell, returns a general overview of the status of the database, detailing disk usage, memory use, connection, journaling, and index access. The command returns quickly and does not impact MongoDB performance.

serverStatus outputs an account of the state of a MongoDB instance. This command is rarely run directly. In most cases, the data is more meaningful when aggregated, as one would see with monitoring tools including MongoDB Cloud Manager and Ops Manager. Nevertheless, all administrators should be familiar with the data provided by serverStatus.

dbStats

The dbStats command, or db.stats() from the shell, returns a document that addresses storage use and data volumes. The dbStats reflect the amount of storage used, the quantity of data contained in the database, and object, collection, and index counters.

Use this data to monitor the state and storage capacity of a specific database. This output also allows you to compare use between databases and to determine the average document size in a database.

collStats

The collStats or db.collection.stats() from the shell that provides statistics that resemble dbStats on the collection level, including a count of the objects in the collection, the size of the collection, the amount of disk space used by the collection, and information about its indexes.

replSetGetStatus

The replSetGetStatus command (rs.status() from the shell) returns an overview of your replica set’s status. The replSetGetStatus document details the state and configuration of the replica set and statistics about its members.

Use this data to ensure that replication is properly configured, and to check the connections between the current host and the other members of the replica set.

Third Party Tools

A number of third party monitoring tools have support for MongoDB, either directly, or through their own plugins.

Self Hosted Monitoring Tools

These are monitoring tools that you must install, configure and maintain on your own servers. Most are open source.

Tool Plugin Description
Ganglia mongodb-ganglia Python script to report operations per second, memory usage, btree statistics, master/slave status and current connections.
Ganglia gmond_python_modules Parses output from the serverStatus and replSetGetStatus commands.
Motop None Realtime monitoring tool for MongoDB servers. Shows current operations ordered by durations every second.
mtop None A top like tool.
Munin mongo-munin Retrieves server statistics.
Munin mongomon Retrieves collection statistics (sizes, index sizes, and each (configured) collection count for one DB).
Munin munin-plugins Ubuntu PPA Some additional munin plugins not in the main distribution.
Nagios nagios-plugin-mongodb A simple Nagios check script, written in Python.
SPM Performance Monitoring MongoDB Docker Agent Monitoring, Anomaly Detection and Alerting SPM monitors all key MongoDB metrics together with infrastructure incl. Docker and other application metrics e.g. Node.js, Java, NGINX, Apache, HAProxy or Elasticsearch. SPM is available On Premises and in the Cloud (SaaS) and provides correlation of metrics and logs.

Also consider dex, an index and query analyzing tool for MongoDB that compares MongoDB log files and indexes to make indexing recommendations.

See also

Ops Manager, an on-premise solution available in MongoDB Enterprise Advanced.

Hosted (SaaS) Monitoring Tools

These are monitoring tools provided as a hosted service, usually through a paid subscription.

Name Notes
MongoDB Cloud Manager MongoDB Cloud Manager is a cloud-based suite of services for managing MongoDB deployments. MongoDB Cloud Manager provides monitoring, backup, and automation functionality. For an on-premise solution, see also Ops Manager, available in MongoDB Enterprise Advanced.
VividCortex VividCortex provides deep insights into MongoDB production database workload and query performance – in one-second resolution. Track latency, throughput, errors, and more to ensure scalability and exceptional performance of your application on MongoDB.
Scout Several plugins, including MongoDB Monitoring, MongoDB Slow Queries, and MongoDB Replica Set Monitoring.
Server Density Dashboard for MongoDB, MongoDB specific alerts, replication failover timeline and iPhone, iPad and Android mobile apps.
Application Performance Management IBM has an Application Performance Management SaaS offering that includes monitor for MongoDB and other applications and middleware.
New Relic New Relic offers full support for application performance management. In addition, New Relic Plugins and Insights enable you to view monitoring metrics from Cloud Manager in New Relic.
Datadog Infrastructure monitoring to visualize the performance of your MongoDB deployments.
SPM Performance Monitoring Monitoring, Anomaly Detection and Alerting SPM monitors all key MongoDB metrics together with infrastructure incl. Docker and other application metrics, e.g. Node.js, Java, NGINX, Apache, HAProxy or Elasticsearch. SPM provides correlation of metrics and logs.

Process Logging

During normal operation, mongod and mongos instances report a live account of all server activity and operations to either standard output or a log file. The following runtime settings control these options.

  • quiet. Limits the amount of information written to the log or output.
  • verbosity. Increases the amount of information written to the log or output. You can also modify the logging verbosity during runtime with the logLevel parameter or the db.setLogLevel() method in the shell.
  • path. Enables logging to a file, rather than the standard output. You must specify the full path to the log file when adjusting this setting.
  • logAppend. Adds information to a log file instead of overwriting the file.

Note

You can specify these configuration operations as the command line arguments to mongod or mongos

For example:

mongod -v --logpath /var/log/mongodb/server1.log --logappend

Starts a mongod instance in verbose mode, appending data to the log file at /var/log/mongodb/server1.log/.

The following database commands also affect logging:

Log Redaction

New in version 3.4: Available in MongoDB Enterprise only

A mongod running with security.redactClientLogData redacts messages associated with any given log event before logging, leaving only metadata, source files, or line numbers related to the event. security.redactClientLogData prevents potentially sensitive information from entering the system log at the cost of diagnostic detail.

For example, the following operation inserts a document into a mongod running without log redaction. The mongod has systemLog.component.command.verbosity set to 1:

db.clients.insertOne( { "name" : Joe, "PII" : "Sensitive Information" } )

This operation produces the following log event:

2017-06-09T13:35:23.446-0400 I COMMAND  [conn1] command internal.clients
   appName: "MongoDB Shell"
   command: insert {
      insert: "clients",
      documents: [ {
            _id: ObjectId('593adc5b99001b7d119d0c97'),
            name: "Joe",
            PII: " Sensitive Information"
         } ],
      ordered: true
   }
   ...

A mongod running with security.redactClientLogData performing the same insert operation produces the following log event:

2017-06-09T13:45:18.599-0400 I COMMAND  [conn1] command internal.clients
   appName: "MongoDB Shell"
   command: insert {
      insert: "###", documents: [ {
         _id: "###", name: "###", PII: "###"
      } ],
      ordered: "###"
   }

Use redactClientLogData in conjunction with encryption to assist compliance with regulatory requirements.

Diagnosing Performance Issues

As you develop and operate applications with MongoDB, you may want to analyze the performance of the database as the application. MongoDB Performance discusses some of the operational factors that can influence performance.

Replication and Monitoring

Beyond the basic monitoring requirements for any MongoDB instance, for replica sets, administrators must monitor replication lag. “Replication lag” refers to the amount of time that it takes to copy (i.e. replicate) a write operation on the primary to a secondary. Some small delay period may be acceptable, but two significant problems emerge as replication lag grows:

  • First, operations that occurred during the period of lag are not replicated to one or more secondaries. If you’re using replication to ensure data persistence, exceptionally long delays may impact the integrity of your data set.

  • Second, if the replication lag exceeds the length of the operation log (oplog) then MongoDB will have to perform an initial sync on the secondary, copying all data from the primary and rebuilding all indexes. This is uncommon under normal circumstances, but if you configure the oplog to be smaller than the default, the issue can arise.

    Note

    The size of the oplog is only configurable during the first run using the --oplogSize argument to the mongod command, or preferably, the oplogSizeMB setting in the MongoDB configuration file. If you do not specify this on the command line before running with the --replSet option, mongod will create a default sized oplog.

    By default, the oplog is 5 percent of total available disk space on 64-bit systems. For more information about changing the oplog size, see the Change the Size of the Oplog

For causes of replication lag, see Replication Lag.

Replication issues are most often the result of network connectivity issues between members, or the result of a primary that does not have the resources to support application and replication traffic. To check the status of a replica, use the replSetGetStatus or the following helper in the shell:

rs.status()

The replSetGetStatus reference provides a more in-depth overview view of this output. In general, watch the value of optimeDate, and pay particular attention to the time difference between the primary and the secondary members.

Sharding and Monitoring

In most cases, the components of sharded clusters benefit from the same monitoring and analysis as all other MongoDB instances. In addition, clusters require further monitoring to ensure that data is effectively distributed among nodes and that sharding operations are functioning appropriately.

See also

See the Sharding documentation for more information.

Config Servers

The config database maintains a map identifying which documents are on which shards. The cluster updates this map as chunks move between shards. When a configuration server becomes inaccessible, certain sharding operations become unavailable, such as moving chunks and starting mongos instances. However, clusters remain accessible from already-running mongos instances.

Because inaccessible configuration servers can seriously impact the availability of a sharded cluster, you should monitor your configuration servers to ensure that the cluster remains well balanced and that mongos instances can restart.

MongoDB Cloud Manager and Ops Manager monitor config servers and can create notifications if a config server becomes inaccessible. See the MongoDB Cloud Manager documentation and Ops Manager documentation for more information.

Balancing and Chunk Distribution

The most effective sharded cluster deployments evenly balance chunks among the shards. To facilitate this, MongoDB has a background balancer process that distributes data to ensure that chunks are always optimally distributed among the shards.

Issue the db.printShardingStatus() or sh.status() command to the mongos by way of the mongo shell. This returns an overview of the entire cluster including the database name, and a list of the chunks.

Stale Locks

To check the lock status of the database, connect to a mongos instance using the mongo shell. Issue the following command sequence to switch to the config database and display all outstanding locks on the shard database:

use config
db.locks.find()

The balancing process takes a special “balancer” lock that prevents other balancing activity from transpiring. In the config database, use the following command to view the “balancer” lock.

db.locks.find( { _id : "balancer" } )

Changed in version 3.4: Starting in 3.4, the primary of the CSRS config server holds the “balancer” lock, using a process id named “ConfigServer”. This lock is never released. To determine if the balancer is running, see Check if Balancer is Running.

Storage Node Watchdog

New in version 3.4.7.

Note

Available only in MongoDB Enterprise. Not available on macOS.

The Storage Node Watchdog monitors the filesystems used by MongoDB to detect unresponsive conditions.

The Storage Node Watchdog can be enabled with the watchdogPeriodSeconds parameter on a mongod.

When enabled, the Storage Node Watchdog monitors the following directories:

If any of the filesystems containing these directories become unresponsive, the Storage Node Watchdog terminates the mongod and exits with a status code of 61. If the mongod is serving as the primary, terminating initiates failover allowing another member to become primary.

Once a mongod has terminated, it may not be possible to cleanly restart it on the same machine.

The maximum time the Storage Node Watchdog can take to detect an unresponsive filesystem and terminate is nearly twice the value of watchdogPeriodSeconds.

© MongoDB, Inc 2008-present. MongoDB, Mongo, and the leaf logo are registered trademarks of MongoDB, Inc.