cbbackup tool

The cbbackup tool creates a copy of data from an entire running cluster, an entire bucket, a single node, or a single bucket on a single functioning node.

Description

The backup process writes a copy of data onto disk. To create a backup using cbbackup, the node or cluster needs to be functioning in order.

The cbbackup, cbrestore, and cbtransfer tools do not communicate with external IP addresses for server nodes outside of a cluster. Backup, restore, or transfer operations are performed on data from a node within a Couchbase cluster. They only communicate with nodes from a node list obtained within a cluster. This also means that if Couchbase Server is installed with a default IP address, an external hostname cannot be used to access it.

This tool has several different options which you can use to:

  • Backup all buckets in an entire cluster
  • Backup one named bucket in a cluster
  • Backup all buckets on a node in a cluster
  • Backup one named buckets on a specified node
Tip: We recommended that cbbackup output be generated on a file system local to the server. Specifically, back up to a dedicated partition. A dedicated partition prevents the backup from from filling the Couchbase data stores and operating system partitions.
CAUTION:
Avoid routing the cbbackup output to remote share file systems (NFS). This is because cbbackup files are based on sqlite format and sqlite-formatted file have issues being written to remote file systems.

This tool is the following directories:

Operating system Location
Linux /opt/couchbase/bin/cbbackup
Windows C:\Program Files\Couchbase\Server\bin\cbbackup
Mac OS X /Applications/Couchbase Server.app/Contents/Resources/couchbase-core/bin/cbbackup

CLI commands and parameters

The format of the cbbackup command is:

cbbackup [options] [source] [backup-dir]

Where:

  • [options]

    Same options available for cbtransfer tool.

  • [source]

    Source for the backup. This can be either a URL of a node when backing up a single node or the cluster, or a URL specifying a directory where the data for a single bucket is located.

  • [backup-dir]

    The directory for the backup files to be stored. Either the directory must exist, and be empty, or the directory will be created. The parent directory must exist.

CLI command and parameters

The following are the command options:

Table 1. cbrestore options
Parameters Description
-h, –help Command line help.
-b BUCKET_SOURCE, --bucket-source=BUCKET_SOURCE Single named bucket from source cluster to transfer. If the backup directory only contains a single bucket, then that bucket is automatically used.
--single-node Use a single server node from the source only, not all server nodes from the entire cluster. This single server node is defined by the source URL.
-m MODE, --mode=MODE  
-i ID, –id=ID Transfer only items that match a vbucket ID.
-k KEY, –key=KEY Transfer only items with keys that match a regexp.
-n, –dry-run No actual transfer. Just validate parameters, files, connectivity and configurations.
-u USERNAME, –username=USERNAME REST username for source cluster or server node.
-p PASSWORD, –password=PASSWORD REST password for cluster or server node.
-t THREADS, –threads=THREADS Number of concurrent workers threads performing the transfer.
-v, –verbose Verbose logging. More v's provide more verbosity. Max: -vvv.
-x EXTRA, –extra=EXTRA Provide extra, uncommon configuration parameters. Comma-separated key=val(key-val)* pairs.

The following are extra, specialized command options with the cbbackup -x parameter.

Table 2. cbbackup -x options
-x options Description
backoff_cap=10 Maximum backoff time during the rebalance period.
batch_max_bytes=400000 Transfer this # of bytes per batch.
batch_max_size=1000 Transfer this # of documents per batch.
cbb_max_mb=100000 Split backup file on destination cluster if it exceeds the MB.
conflict_resolve=1 By default, disable conflict resolution.
data_only=0 For value 1, transfer only data from a backup file or cluster.
design_doc_only=0 For value 1, transfer only design documents from a backup file or cluster. Default: 0.
max_retry=10 Max number of sequential retries if the transfer fails.
mcd_compatible=1 For value 0, display extended fields for stdout output.
nmv_retry=1 0 or 1, where 1 retries transfer after a NOT_MY_VBUCKET message. Default: 1.
recv_min_bytes=4096 Amount of bytes for every TCP/IP batch transferred.
rehash=0 For value 1, rehash the partition id's of each item. This is required when transferring data between clusters with different number of partitions, such as when transferring data from an Mac OS X server to a non-Mac OS X cluster.
report=5 Number batches transferred before updating progress bar in console.
report_full=2000 Number batches transferred before emitting progress information in console.
seqno=0 By default, start seqno from beginning.
try_xwm=1 Transfer documents with metadata. Default: 1. Value of 0 is only used when transferring from 1.8.x to 1.8.x.
uncompress=0 For value 1, restore data in uncompressed mode.

Syntax

The following is the basic syntax:

cbrestore [options] [backup-dir] [destination]

The following syntax examples include a full backup and two incremental backups for a cluster.:


cbbackup http://HOST:8091 /backup-42
cbbackup http://HOST:8091 /backup-42
cbbackup http://HOST:8091 /backup-42
			

The following syntax examples include a full backup, two differential backups, and one accumulative backup for a single node.


cbbackup couchbase://HOST:8091 /backup-43 [-m full] --single-node
cbbackup couchbase://HOST:8091 /backup-43 [-m diff] --single-node
cbbackup couchbase://HOST:8091 /backup-43 [-m diff] --single-node
cbbackup couchbase://HOST:8091 /backup-43 -m accu --single-node
         
Note: After backing up and restoring a cluster, be sure to rebuild your indexes.

Example: Backing up clusters

An entire cluster can be backed up. This includes all of the data buckets and data at all nodes and all design documents. To backup an entire cluster and all buckets for that cluster:

cbbackup http://HOST:8091 ~/backups \
          -u Administrator -p password

Where ~/backups is the directory where you want to store the data. When this operation is performed, be aware that cbbackup creates the following directory structure and files in the ~/backups directory assuming there two buckets in the cluster named my_name and sasl and two nodes N1 and N2 :

~/backups
        bucket-my_name
            N1
            N2
        bucket-sasl
            N1
            N2

Where bucket-my_name and bucket-sasl are directories containing data files and where N1 and N2 are two sets of data files for each node in the cluster.

Example: Backing up a single bucket

To backup a single bucket in a cluster:

cbbackup http://HOST:8091 /backups/backup-20120501 \
  -u Administrator -p password \
  -b default

In this case, the default bucket in the cluster is specified and the data from the default bucket is backed up.

Example: Backing up multiple buckets

To backup all data from multiple buckets on a single node:

> cbbackup http://HOST:8091 /backups/ \
  -u Administrator -p password \
  --single-node

To backup data from a single bucket on a single node:

cbbackup http://HOST:8091 /backups \
  -u Administrator -p password \
  --single-node \
  -b bucket_name

Example: Backing up keys

To specify the keys that are backed up using the - k option. For example, to backup all keys from a bucket with the prefix ‘object’:

> cbbackup http://HOST:8091 /backups/backup-20120501 \
  -u Administrator -p password \
  -b bucket_name \
  -k '^object.*'