Server warmup

Whenever the Couchbase server is restarted or data is restored to a server instance, the server undergoes a warmup process before data requests can be handled.

During server warmup, the server loads data persisted on disk into RAM. Couchbase Server provides an optimized warmup process which loads data sequentially from disk into RAM, it divides the data to be loaded and handles it in multiple phases. After the warmup process completes, the data is available for clients to read and write. The time needed for server warmup depends on system size, system configuration and the amount of data persisted in the system.

Note: Couchbase Server is able to begin serving data before it has actually loaded all the keys and data from vBuckets.

Couchbase server identifies items that are frequently used, prioritizes them, and loads them before sequentially loading the remainer of the data. The frequently-used items are prioritized in an access log. The server pre-fetches a list of the most frequently accessed keys and then fetches these documents before any other items from disk.

The server runs a configurable scanner process that determines which keys are most frequently-used. The scanner process is pre-set and is configurable. The command-line tool, cbepctl flush_param is used to change the initial time and interval for the scanner process. For example, you might want to configure the scanner process to be run during a specific time period when certain keys need to be identified and made available sooner.

The server can also switch into a ready mode before it has actually retrieved all documents for keys into RAM, therfore, data can be served before all stored items are loaded. Switching into ready mode is a configurable setting so that server warmup time can be adjusted.

The following describes the initial warmup phases for the Couchbase Server. In these first phase, the server begins to fetch all keys and metadata from disk. Then the server accesses the log information that it needs to retrieve the most used keys:

  1. Initialize - At this phase, the server does not have any data that it can serve yet. The server starts populating a list of all vBuckets stored on disk by loading the recorded, initial state of each vBucket.
  2. Key Dump - The server begins pre-fetching all keys and metadata from disk based on items in the vBucket list.
  3. Check Access Logs - The server then reads a single cached access log which indicates which keys are frequently accessed. The server generates and maintains this log on a periodic basis and it is configurable. If this log exists, the server first loads items based on this log before it loads other items from disk.

Once Couchbase Server has the information about keys and has read in any access log information, it loads documents based on the following criteria:

  • Loading based on Access Logs - Couchbase Server loads documents into memory based on the frequently-used items identified in the access log.

  • Loading Data - If the access log is empty or is disabled, the server sequentially loads documents for each key based on the vBucket list.

Couchbase Server is able to serve information from RAM when one of the following conditions is met during warmup:

  • The server has finished loading documents for all keys listed in the access log
  • The server has finished loading documents for every key stored on disk for all vBuckets
  • The percentage of documents loaded into memory is greater than, or equal to, the setting for the cbepctl ep_warmup_min_items_threshold parameter
  • If total % of RAM filled by documents is greater than, or equal to, the setting for the cbepctl ep_warmup_min_memory_threshold parameter
  • If total RAM usage by a node is greater than or equal to the setting for mem_low_wat.

When the server reaches one of these states, known as the run level, the server stops loading documents for the remaining keys. and loads the remaining documents from disk into RAM as a background data fetch.