[ previous ] [ Contents ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] [ 7 ] [ 8 ] [ 9 ] [ 10 ] [ 11 ] [ 12 ] [ 13 ] [ 14 ] [ 15 ] [ 16 ] [ 17 ] [ 18 ] [ next ]



Debian GNU/Linux System Administrator's Manual
Chapter 12 - Backup and Restore


author = Tapio Lehtonen

topics = why backup, what to backup, how to backup, backup devices and media


12.1 Why backup?

Backups are needed in case a file or a group of files is lost. The reasons for losing files include

Backups help in all the above situations. In addition, it may be good to have access to older versions of files, for example a configuration file worked a week ago, but since then it has been changed and nobody remembers how, its just not working anymore.

There are other solutions, and they are good to have if you can afford them. These include

These help, but if there is anything you do not want to lose on the computer, make sure there are backups and they can be restored.

This chapter is more of a general nature than the others, which are specific to the Debian GNU/Linux. There are so many different backup devices and backup software, that it is difficult to go into details without assuming for example a SCSI tape drive and GNU tar used to write backups. Chapter Floppy, Section 12.3.1 gives three detailed examples, use them as a guide in doing backups to other kinds of media.


12.2 What to backup?

If there is room on the backup media, and time limits permit running backups long enough, it probably is wisest to back up everything. You may skip /tmp or other places where it is known there are only temporary files that nobody wants to backup.

If space or time limits place restrictions, consider not backing up the following:


12.3 Backup devices and media

You need some media to store the backups. It is preferable to use removable media, to store the backups away from the computer and to get "unlimited" storage for backups.

If the backups are on-line, they can be wiped out by mistake. If the backups are on the same disk as the original data, they do not help at all if the disk fails and is not readable anymore. If the backup media is cheap, it is possible to take a backup every day and store them indefinitely.

The following subsections discuss different kinds of backup media. As a hands-on example, a floppy is used to back up directory trees in different ways. Use these examples as a guideline when using other medias.


12.3.1 Floppy

Floppy disks are cheap, and on PC computers there usually is a floppy disk drive. On the other hand, it is not very fast to write to a floppy, and the capacity of 1.4MBytes is not very much. If the backup does not fit on one media, taking backups becomes an arduous task, what with having to stand by and change the media every now and then.

However, if the data to be backed up fits in one floppy, they are a reasonable alternative. Compressing the data usually means it uses about 50% of the original size. Thus, you can expect to get almost 3MBytes to one 1.4M floppy disk with compression.

Examples use tar, because it is available on almost all Unix versions and also on some other operating systems, it can preserve file ownerships and date stamps and write directly to device or to a file.

There are different ways to use the floppy with tar:

  1. Create a Linux filesystem on the floppy, mount, write like to any Linux disk and unmount the floppy,

  1. write directly to the device /dev/fd0 and

  1. use the floppy with DOS file system, and copy the tar file there with mcopy (see info file mtools with command info mtools).

These three methods correspond to three different classes of media:

  1. Random access or direct access, mostly disks. The media is like any disk or directory tree, it is possible to do ls, cp and other commands accessing files. It is easy to restore one file from the backup media by simply copying it back.

  1. Serial access, like a tape drive. Reading or writing the media starts from the beginning and goes to the end. It is not possible to start directly in the middle.

  1. Media that is a DOS file system. This is readable on all kinds of operating system, which is useful if you need to read the backup on some other kind of computer.


12.3.1.1 Backup example 1, suitable for disk media

Here is an example to backup the ~/Work/Debian-doc directory tree to floppy. First check how large the directory tree is:

     $ cd ~/Work
     tale@dilbert ~/Work
     $ du -sk Debian-doc
     1005    Debian-doc
     tale@dilbert ~/Work
     $

This shows 1.005MBytes, so it should fit on one floppy.

This example uses the floppy as removable media with Linux filesystem.

Now a minix filesystem is created on the floppy. This is the filesystem Linux uses for floppies and other small media. Note, that all data on the media is lost when it is formatted. The first floppy drive is device /dev/fd0.

     dilbert# mkfs -t minix -c /dev/fd0 1440
     480 inodes
     1440 blocks
     Firstdatazone=19 (19)
     Zonesize=1024
     Maxsize=268966912
     
     160 ...320 ...480 ...640 ...800 ...960 ...1120 ...1296 ...dilbert#

Now the floppy disk is mounted and backup is run and tested. Note, that the mount point /A must already exist (see mount(8). To allow an ordinary user tale write access to the filesystem on this floppy, the owner and group owner are changed.

     dilbert# mount -t minix /dev/fd0 /A
     dilbert# chown tale.tale /A
     dilbert# ls -ld /A
     drwxr-xr-x   3 tale     tale         1024 Dec  3 21:21 /A
     dilbert#

It is useful the check the floppy is mounted and there is available space, for example with command df.

     $ df -h
     Filesystem            Size  Used  Avail  Capacity Mounted on
     /dev/sda5              99M   22M    72M     23%   /
     /dev/sda6             964M  565M   349M     62%   /usr
     /dev/sda7             190M   33M   147M     18%   /var
     /dev/sda8             2.7G  1.3G   1.2G     53%   /export
     /dev/sdb2             3.8G  2.4G   1.3G     65%   /export2
     dilbert:/export/home  2.7G  1.3G   1.2G     53%   /amd/dilbert/export/home
     /dev/fd0              1.4M  1.0K   1.4M      0%   /A
     tale@dilbert ~/Work
     $

Now tar is used to copy the whole directory tree to the floppy. It is better to use tar, it preserves file ownerships and permissions. If you try to use cp -r you will notice the backup is not identical.

     $ cd Work 
     tale@dilbert ~/Work
     $ tar cf - ./Debian-doc | ( cd /A ; tar xpBf - ) 
     tale@dilbert ~/Work
     $ ls -l /A
     total 1
     drwxr-xr-x   6 tale     232           320 Dec  3 20:41 Debian-doc
     tale@dilbert ~/Work
     $

You can also do ls -lR /A to see that all the files are there.

Here we restore the backup to /tmp/Debian-doc.

     $ cd /A  
     tale@dilbert /A
     $ ls -l
     total 1
     drwxr-xr-x   6 tale     232           320 Dec  3 20:41 Debian-doc
     tale@dilbert /A
     $ tar cf - ./Debian-doc | ( cd /tmp ; tar xpBf - )
     tale@dilbert /tmp
     $ ls -l
     total 7512
     drwxr-xr-x   6 tale     tale         1024 Dec  3 20:41 Debian-doc
     -rw-------   1 root     root       599648 Dec  3 22:25 foo.foo
     drwx------   2 tale     tale         1024 Dec  2 20:58 orbit-tale
     -rw-rw----   1 tale     tale      4054572 Nov 24 21:53 tale
     tale@dilbert /tmp
     $ du --summarize --kilobytes Debian-doc/
     1005    Debian-doc
     tale@dilbert /tmp
     $

It looks like the backup is on the floppy and we could even read it back. It is worth checking your backup method works, and the backup can indeed be restored.

The next command unmounts the floppy. This is very important!. If you mount a removable media, it must be unmounted before it is removed from the drive.

     dilbert# umount /A
     dilbert#

The above method is usable with other kinds of removable disk drives, for example high capacity floppies.


12.3.1.2 Backup example 2, suitable for tape media

As another example, we backup /etc directly to the /dev/fd0 device:

     dilbert# cd /
     dilbert# tar czf /dev/fd0 ./etc
     dilbert# tar tzf /dev/fd0 | head
     ./etc/
     ./etc/modules
     ./etc/xemacs/
     ./etc/xemacs/site-start-19.d/
     ./etc/xemacs/site-start-19.d/50dpkg-dev.el
     ./etc/init.d/
     ./etc/init.d/network
     ./etc/init.d/modutils
     ./etc/init.d/kerneld
     ./etc/init.d/kdm
     Broken pipe
     dilbert#

We could read the listing (option t for tar) from the floppy, so we can assume the backup is OK. Note, that if you plan to read this floppy back later, make sure you remember how the floppy was written. If you forget it was written with tar and with compression, you spend a lot of time figuring it out. This same applies to all media that you store for any longer period of time, and especially if you send the media to someone else.

The above method is usable also with tape drives. Replace the device name /dev/fd0 with device name for the tape drive, and you can use tar as Tape ARchiver.


12.3.1.3 Backup example 3, suitable for disk media with DOS file system

If you have or want to use DOS formatted floppy disks, it is possible to use them like in example 1, mount them with flag -t msdos to inform the mount command. But in this example, we us DOS floppies with the mtools -commands.

The backup is not written directly to floppy, it is first created on /tmp directory and copied from there to floppy with command mcopy. For more information, use info mtools.

     $ pwd
     /amd/dilbert/export/home/tale/Work
     tale@dilbert /amd/dilbert/export/home/tale/Work
     $ du -sk Debian-doc
     2399    Debian-doc
     tale@dilbert /amd/dilbert/export/home/tale/Work
     $ tar czf /tmp/Debian-doc.tgz ./Debian-doc
     tale@dilbert /amd/dilbert/export/home/tale/Work
     $ ls -l /tmp/Debian-doc.tgz 
     -rw-rw-r--   1 tale     tale       609296 Dec  5 04:56 /tmp/Debian-doc.tgz
     tale@dilbert /amd/dilbert/export/home/tale/Work
     $ mcopy /tmp/Debian-doc.tgz a:
     tale@dilbert /amd/dilbert/export/home/tale/Work
     $ mdir a:
      Volume in drive A is LINUX BOOT 
      Volume Serial Number is 2410-07EF
     Directory for A:/
     
     DEBIAN~1 TGZ    609296 12-05-1998   4:58  Debian-doc.tgz
             1 file              609 296 bytes
                                 847 872 bytes free
     
     tale@dilbert /amd/dilbert/export/home/tale/Work
     $

This method can be used when the backup needs to be read back on some other computer, not necessarily running Linux. This is also useful for sending files to some poor soul not running Linux. In this case, it is better not to use tar, programs like zip, gzip and zoo are available on most operating systems.


12.3.2 High Capacity Floppies

High capacity floppies are

All of these are about floppy disk size, store 100MBytes to 200MBytes and are faster and more expensive than floppy drives. They are connected to EIDE port, Parallel port, SCSI or USB. LS-120 and Sony SuperFloppy can read and write ordinary 1.4MByte floppy disks.

You can use these things like in the above examples where a floppy disk was used, but you have to install the devices and the device driver software before they can be used. Then the device name depends on what kind of connection the thing uses.

There is more info on using the above high capacity floppies on Linux in the HOWTO documents. (reference to HOWTOs???)

Somebody with experience from the above devices: please confirm my guessing above or send info on how they can be used.


12.3.3 CD-R and CD-RW

Meta: Capacity <= 640MBytes, speed 150 -- 600KBits/sek. Special software needed to burn the CD. DVD-RAM

CD writers can be used as backup devices. Writable CD media is either writable exactly once (CD-R), or erasable and rewritable (CD-RW). CD-R disks can be read on ordinary CD drives, but CD-RW disks need Multi-Read capability from the reader. This is good to know if you plan to read the CD back on some other computer.

Assuming the CD writer is installed and configured correctly, and you have the necessary software to write to the CD, taking backups is best done with the first method in the floppy disk example above, i.e. creating a Linux file system on the CD. Since the CD is a disk, i.e. a random access device, using it this way is easy. Just mount it and copy files or whole directory trees there.

Restoring is also straightforward, since the CD can be examined with ordinary file system commands like ls, and it is easy to copy a single file back. You can also compare the files in the backup to files in hard disk with diff for example.

Problems with CD-R are their write once -nature. They need to be written all in one go, and then closed. After closing, it is not possible to modify the CD, so if there is something wrong there it has to be thrown away.

CD-RW can be erased, but my understanding is the whole disk must be erased. In addition, looks like formatting a CD-RW takes about one hour.

Looks like the program to write CD's is X-CD-Roast, available as Debian GNU/Linux package xcdroast. More information from X-CD-Roast Webpage


12.3.4 Tapes

Meta: Popular backup media. Wide range of speeds, capacities and cost.

Tape drives are popular backup devices. The media is relatively cheap per gigabyte, and tape capacities go up to several tens of gigabytes. On the other hand, the tape drives may be expensive and write speeds slower than disks.

Tape drives with SCSI connector should work with Linux. So called floppy tape drives that connect to the floppy disk interface may work if the ftape driver supports the particular model.

Writing to SCSI tape drives goes like in example 2 in floppy disk chapter.

I'm not sure about floppy tapes, have to read the ftape docs.


12.4 Backup methods and software

Backup methods include simply copying files to another media, using dd, tar or similar program to create an archive and using special backup programs.


12.4.1 Network backups

In an enterprise environment there may be a backup server running some network backup software. If there are GNU/Linux clients available for that software, its possible to install them, configure the client machine on the backup server and start taking backups over the network. This is a low cost solution. If the backup server is already there, GNU/Linux clients are sometimes free to download.


12.4.1.1 Installing EMC NetWorker Client

EMC NetWorker is a backup system formerly known as Legato. EMC supplies NetWorker Client for GNU/Linux, but with almost no technical support. The applications are available in RPM binary packages from the Legato FTP Site. The tarball will uncompress to several RPM packages. A system that will only send files to the backup server will need lgtoclnt-7.3-1.i686.rpm and lgtoman-7.3-1.i686.rpm.

The client package will allow the backup server to connect to your system and to request the files that need to be backup to. The configuration is almost all done on the backup server. The client system only need to know which backup server to allow. Your contract should allow you install the client packages on your system as long as you have the server licence but please check with your EMC representative if you have doubts.

To install these packages on a Debian GNU/Linux system, they must be converted to Debian .deb format. The files supplied by Legato are relocatable .rpm files, and alien version 6.27 and smaller can not correctly convert these. Use alien version 6.28 or later.

Converting the packages goes like this:

     vendredi:/tmp# alien --to-deb lgtoclnt-7.3-1.i686.rpm
     lgtoclnt_7.3-2_i386.deb generated
     vendredi:/tmp# alien --to-deb lgtoman-7.3-1.i686.rpm
     lgtoman_7.3-2_i386.deb generated

You can then install the packages like this

     # dpkg --install lgtoclnt-7.3-1.i686.deb
     # dpkg --install lgtoman-7.3-1.i686.deb

The conversion from RPMs is not perfect and you will have to perform a few steps by hand to have everything running.


12.4.1.2 Setting up and Configuring

To get backups with Legato, there must be nsrexecd running on the backup client host. This process communicates with the Legato server. On a client only system, other Legato prosesses are not needed, nsrexecd starts them as needed.

You need to tell nsrexecd from which backup server it can accept connection. This can be done from the command line or from a resource file. The preferred way is from a resource file:

     # mkdir -p /nsr/res
     # echo "backup-server.example.com" > /nsr/res/servers
     # echo "other.example.net"        >> /nsr/res/servers

Note that NetWorker uses a non lsb compliant directory /nsr/. There is no documented way to tell it to look in /etc/.

If you are using a firewall, you need to open the NetWorker ports. NetWorker uses remote procedure calls based on Sun RPCs with its own portmapper. You need to open the portmapper ports (7937:7938) and the RPC port range (10001:10100). With shorewall you would do it by putting those rules in /etc/shorewall/rules

     # EMC NetWorker (Backups)
     ACCEPT          net fw udp      7937:7938
     ACCEPT          net fw tcp      7937:7938
     ACCEPT          net fw tcp     10001:10100

To check the installation, start nsrexecd manually, check that it is running and test what the backup server can do with this new client.

     vendredi:/tmp/Legato# /usr/sbin/nsrexecd
     vendredi:/tmp/Legato# ps aux | grep nsr
     root      4031  0.5  0.6  1132   768  ?  S    15:48   0:00 /usr/sbin/nsrexecd 
     root      4033  0.0  0.2   848   344  p2 S    15:48   0:00 grep nsr 
     vendredi:/tmp/Legato#

Next go to the backup server, and do a test run, where nothing is actually saved to tape, but Legato server contacts the client and probes the file systems. This way you can check most of the functionality and can see what Legato would do when started for real.

     root@lehmus:/# savegrp -n -p -v -P hplp -c vendredi  Linux_Clients
     vendredi:All                              level=incr
     01/21/99 16:26:17 savegrp: Run up to 12 clients in parallel
     01/21/99 16:26:17 savegrp: vendredi:probe                                started
     savefs -s lehmus -g Linux_Clients -p -n -l full -R -v 
     01/21/99 16:26:20 savegrp: vendredi:probe succeeded.
     rcmd vendredi, user root: `savefs -s lehmus -g Linux_Clients -p -n -l full -R -v'
     nsrexec: authtype nsrexec
     type: NSR client description;
     pools supported: Yes;
     remote user: root;
     groups: daemon, root, bin, sys, adm, tty, disk, lp, mail, news, daemon;
     arch: i586;
     client OS type: Linux;
     CPUs: 1;
     IP address: 172.19.13.36;
     machine type: desktop;
     MB used: 828;
     NetWorker version: Utah.Build.MJFeral_8_11_98_SNAP;
     OS: Linux 2.0.34;
     version: Utah.Build.MJFeral_8_11_98_SNAP;
     save set: path=/, level=full, diskno=0, max_sessions=3, stype=save,\
     path=/var, level=full, diskno=0, max_sessions=3, stype=save,\
     path=/export/home, level=full, diskno=0, max_sessions=3, stype=save ;
     parallelism: 8
     --- Probe Summary ---
     
     vendredi:/                            level=full, dn=0, mx=3, vers=pools, p=4
     vendredi:/              level=full, pool=SMT Workstation, save as of Thu Jan  1 03:00:00 GMT+0300 1970
     vendredi:/var                         level=full, dn=0, mx=3, vers=pools, p=4
     vendredi:/var           level=full, pool=SMT Workstation, save as of Thu Jan  1 03:00:00 GMT+0300 1970
     vendredi:/export/home                 level=full, dn=0, mx=3, vers=pools, p=4
     vendredi:/export/home   level=full, pool=SMT Workstation, save as of Thu Jan  1 03:00:00 GMT+0300 1970
     vendredi:index                       level=full, dn=-1, mx=0, vers=pools, p=4
     vendredi:index          level=full, pool=SMT Workstation, save as of Thu Jan  1 03:00:00 GMT+0300 1970
     
     root@lehmus:/#

If the previous tests looked OK, you can set up automatic backups on the server (or ask the backup administrator to do it).

The remaining configuration on the Linux host is to add starting of nsrexecd to the boot scripts, for example like this:

     #! /bin/sh
     #
     # Script to start EMC Networker backup daemon
     
     PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
     DAEMON=/usr/sbin/nsrexecd
     NAME=nsrexecd
     DESC="legato backup daemon"
     NICE=15
     
     test -f $DAEMON || exit 0
     
     set -e
     
     case "$1" in
       start)
         echo -n "Starting $DESC: $NAME"
         start-stop-daemon --start -N $NICE -o --exec $DAEMON
         echo "."
         ;;
       stop)
         echo -n "Stopping $DESC: $NAME"
         start-stop-daemon --stop -o --name $NAME --exec $DAEMON
         echo "."
         ;;
       restart|force-reload)
         echo -n "Restarting $DESC: $NAME"
         start-stop-daemon --stop -o --name $NAME --exec $DAEMON
         sleep 1
         start-stop-daemon --start -N $NICE -o --exec $DAEMON
         echo "."
         ;;
       *)
         echo "Usage: /etc/init.d/$NAME {start|stop|restart|force-reload}"
         exit 1
         ;;
     esac
     
     exit 0

You then have to add the backup service to your startup sequence:

     # update-rc.d networker defaults 90

Now Legato should be setup up properly, and automatic backups run as configured on the server.


12.4.2 Tar et al

Meta: Tar, dump, dd, cpio

Now for some examples.

In Debian GNU/Linux the tar program is GNU tar, which has several extra features. Among them is support for compressing the tar file while it is created. On the average, compression squeezes the file to about 50% of the uncompressed size. Your mileage may vary: files that are already compressed, like *.zip and *.gif files do not compress at all, and some file compress especially well, C source code files can go to 25% of the uncompressed size.

Taking a compressed tar file from /export/home partition goes like this:

     cd /export
     tar czf /tmp/home.tgz ./home

The tar file /tmp/home.tgz can be copied to another disk, or to another computer. If you do not have any backup device, but have two computers with free disk space, take a compressed tar of the most important files and copy the tar to the other computer.


12.4.3 Backup software

Meta: amanda, other backup software in Debian

I do not have time to study these now, contribution would be welcome.


12.5 Types of backup

Meta: Full, incremental, differential, network, dump, level 0--9,

There are different kinds of backups, the following lists some of them:

Full

Full backup means backing up everything.

Incremental

Incremental backup means backing up everything that has changed since last full backup.

Differential

Differential seems to be another name for incremental.

Network

Network backup usually means backing up a client to a backup server, this means the client sends the files to the server and the server writes them to backup medium.

Dump

Dump backups are not ordinary file by file backups. The whole disk partition or file system is "dumped" to the backup medium as is. This means it is also necessary to restore the whole partition or file system at one go. The dump backup may be a disk image, which means it must be restored to a similar disk with same disk geometry and bad blocks in same places. Watch out for this.

Level 0 -- 9

Level 0 to 9 backups are a finer grained version of incremental backups. Level N backup means backing up everything that has changed since a same or lower lever backup.

Meta: Check the backup can be restored, with original file owners, permissions and timestamps.

To be useful, you must be able to restore the backup. Very often not only the contents of file are important, but their time stamps, permissions and owners. Check that you can restore the backup so that all these are preserved.


[ previous ] [ Contents ] [ 1 ] [ 2 ] [ 3 ] [ 4 ] [ 5 ] [ 6 ] [ 7 ] [ 8 ] [ 9 ] [ 10 ] [ 11 ] [ 12 ] [ 13 ] [ 14 ] [ 15 ] [ 16 ] [ 17 ] [ 18 ] [ next ]


Debian GNU/Linux System Administrator's Manual


version $VERSION$, 10 January 2007

Ardo van Rangelrooij [email protected]
Tapio Lehtonen [email protected]
Oliver Elphick - Maintainer [email protected]