Gentoo Linux OpenAFS Guide

1. Overview

This document provides you with all neccessary steps to install an openafs server on Gentoo Linux. Parts of this document are taken from the AFS FAQ and IBM's Quick Beginnings guide on AFS. Well, never reinvent the wheel :)

What is AFS ?

AFS is a distributed filesystem that enables co-operating hosts (clients and servers) to efficiently share filesystem resources across both local area and wide area networks. Clients hold a cache for often used objects (files), to get quicker access to them.

AFS is based on a distributed file system originally developed at the Information Technology Center at Carnegie-Mellon University that was called the "Andrew File System". "Andrew" was the name of the research project at CMU - honouring the founders of the University. Once Transarc was formed and AFS became a product, the "Andrew" was dropped to indicate that AFS had gone beyond the Andrew research project and had become a supported, product quality filesystem. However, there were a number of existing cells that rooted their filesystem as /afs. At the time, changing the root of the filesystem was a non-trivial undertaking. So, to save the early AFS sites from having to rename their filesystem, AFS remained as the name and filesystem root.

What is an AFS cell ?

An AFS cell is a collection of servers grouped together administratively and presenting a single, cohesive filesystem. Typically, an AFS cell is a set of hosts that use the same Internet domain name (like for example gentoo.org) Users log into AFS client workstations which request information and files from the cell's servers on behalf of the users. Users won't know on which server a file which they are accessing, is located. They even won't notice if a server will be located to another room, since every volume can be replicated and moved to another server without any user noticing. The files are always accessable. Well it's like NFS on steroids :)

What are the benefits of using AFS ?

The main strengths of AFS are its: caching facility (on client side, typically 100M to 1GB), security features (Kerberos 4 based, access control lists), simplicity of addressing (you just have one filesystem), scalability (add further servers to your cell as needed), communications protocol.

Where can i get more information ?

Read the AFS FAQ.

Openafs main page is at www.openafs.org.

AFS was originally developed by Transarc which is now owned by IBM. You can find some information about AFS on Transarcs Webpage

2. Documentation

Getting AFS Documentation

You can get the original IBM AFS Documentation. It is very well written and you really want read it if it is up to you to administer a AFS Server.

Code Listing 2.1

# emerge app-doc/afsdoc

3. Client Installation

Preliminary Work

Note: All commands should be written in one line !! In this document they are sometimes wrapped to two lines to make them easier to read.

Note: Unfortunately the AFS Client needs a ext2 partiton for it's cache to run correctly, because there are some locking issues with reiserfs. You need to create a ext2 partition of approx. 200MB (more won't hurt) and mount it to /usr/vice/cache

You should adjust the two files CellServDB and ThisCell before you build the afs client. (These files are in /usr/portage/net-fs/openafs/files)

Code Listing 3.1

 CellServDB:
 >netlabs        #Cell name
 10.0.0.1        #storage
 
 ThisCell:
 netlabs

Warning: Only use spaces inside the CellServDB file. The client will most likely fail if you use TABs.

CellServDB tells your client which server(s) he needs to contact for a specific cell. ThisCell should be quite obvious. Normally you use a name which is unique for your organisation. Your (official) domain might be a good choice.

Building the Client

Code Listing 3.2

# emerge net-fs/openafs

After successful compilation you're ready to go.

Starting afs on startup

The following command will create the appropriate links to start your afs client on system startup.

Warning: You should always have a running afs server in your domain when trying to start the afs client. You're system won't boot until it gets some timeout if your afs server is down. (and this is quite a long long time)

Code Listing 3.3

# rc-update add afs default

4. Server Installation

Building the Server

The following command will install all necessary binaries for setting up a AFS Server and Client.

Code Listing 4.1

# emerge net-fs/openafs

Starting AFS Server

You need to remove the sample CellServDB and ThisCell file first.

Code Listing 4.2

# rm /usr/vice/etc/ThisCell
# rm /usr/vice/etc/CellServDB

Next you will run the bosserver command to initialize the Basic OverSeer (BOS) Server, which monitors and controls other AFS server processes on its server machine. Think of it as init for the system. Include the -noauth flag to disable authorization checking, since you haven't added the admin user yet.

Warning: Disabling authorization checking gravely compromises cell security. You must complete all subsequent steps in one uninterrupted pass and must not leave the machine unattended until you restart the BOS Server with authorization checking enabled. Well this is what the AFS documentation says :)

Code Listing 4.3

# /usr/afs/bin/bosserver -noauth &

Verify that the BOS Server created /usr/vice/etc/CellServDB and /usr/vice/etc/ThisCell

Code Listing 4.4

# ls -al /usr/vice/etc/
-rw-r--r--    1 root     root           41 Jun  4 22:21 CellServDB
-rw-r--r--    1 root     root            7 Jun  4 22:21 ThisCell

Defining Cell Name and Membership for Server Process

Now assign your cells name.

Important: There are some restrictions on the name format. Two of the most important restrictions are that the name cannot include uppercase letters or more than 64 characters. Remember that your cell name will show up under /afs, so you might want to choose a short one.

Note: In the following and every instruction in this guide, for the <server name> argument substitute the full-qualified hostname (such as afs.gentoo.org) of the machine you are installing. For the <cell name> argument substitute your cell's complete name (such as gentoo)

Run the bos setcellname command to set the cell name:

Code Listing 4.5

# /usr/afs/bin/bos setcellname <server name> <cell name> -noauth

Starting the Database Server Process

Next use the bos create command to create entries for the four database server processes in the /usr/afs/local/BosConfig file. The four processes run on database server machines only.

kaserver	The Authentication Server maintains the Authentication Database. This can be replaced by a Kerberos 5 daemon. If anybody want's to try that feel free to update this document :)
buserver	The Backup Server maintains the Backup Database
ptserver	The Protection Server maintains the Protection Database
vlserver	The Volume Location Server maintains the Volume Location Database (VLDB). Very important :)

Code Listing 4.6

# /usr/afs/bin/bos create <server name> kaserver simple 
    /usr/afs/bin/kaserver -cell <cell name> -noauth
# /usr/afs/bin/bos create <server name> buserver simple 
    /usr/afs/bin/buserver -cell <cell name> -noauth
# /usr/afs/bin/bos create <server name> ptserver simple 
    /usr/afs/bin/ptserver -cell <cell name> -noauth
# /usr/afs/bin/bos create <server name> vlserver simple 
    /usr/afs/bin/vlserver -cell <cell name> -noauth

You can verify that all servers are running with the bos status command:

Code Listing 4.7

# /usr/afs/bin/bos status <server name> -noauth
Instance kaserver, currently running normally.
Instance buserver, currently running normally.
Instance ptserver, currently running normally.
Instance vlserver, currently running normally.

Initializing Cell Security

Now we'll initialize the cell's security mechanisms. We'll begin by creating the following two initial entries in the Authentication Database: The main administrative account, called admin by convention and an entry for the AFS server processes, called afs. No user logs in under the identity afs, but the Authentication Server's Ticket Granting Service (TGS) module uses the account to encrypt the server tickets that it grants to AFS clients. This sounds pretty much like Kerberos :)

Enter kas interactive mode

Code Listing 4.8

# /usr/afs/bin/kas -cell <cell name> -noauth
ka> create afs
initial_password:
Verifying, please re-enter initial_password:
ka> create admin
initial_password:
Verifying, please re-enter initial_password:
ka> examine afs

User data for afs
  key (0) cksum is 2651715259, last cpw: Mon Jun  4 20:49:30 2001
  password will never expire.
  An unlimited number of unsuccessful authentications is permitted.
  entry never expires.  Max ticket lifetime 100.00 hours.
  last mod on Mon Jun  4 20:49:30 2001 by $lt;none>
  permit password reuse
ka> setfields admin -flags admin
ka> examine admin
 
User data for admin (ADMIN)
  key (0) cksum is 2651715259, last cpw: Mon Jun  4 20:49:59 2001
  password will never expire.
  An unlimited number of unsuccessful authentications is permitted.
  entry never expires.  Max ticket lifetime 25.00 hours.
  last mod on Mon Jun  4 20:51:10 2001 by $lt;none>
  permit password reuse
ka>

Run the bos adduser command, to add the admin user to the /usr/afs/etc/UserList.

Code Listing 4.9

# /usr/afs/bin/bos adduser <server name> admin -cell <cell name> -noauth

Issue the bos addkey command to define the AFS Server encryption key in /usr/afs/etc/KeyFile

Note: If asked for the input key, give the password you entered when creating the afs entry with kas

Code Listing 4.10

# /usr/afs/bin/bos addkey  <server name> -kvno 0 -cell <cell name> -noauth
    input key:
    Retype input key:

Issue the pts createuser command to create a Protection Database entry for the admin user

Note: By default, the Protection Server assigns AFS UID 1 to the admin user, because it is the first user entry you are creating. If the local password file (/etc/passwd or equivalent) already has an entry for admin that assigns a different UID use the -id argument to create matching UID's

Code Listing 4.11

# /usr/afs/bin/pts createuser -name admin -cell <cell name> [-id <AFS UID>] -noauth

Issue the pts adduser command to make the admin user a member of the system:administrators group, and the pts membership command to verify the new membership

Code Listing 4.12

# /usr/afs/bin/pts adduser admin system:administrators -cell <cell name> -noauth
# /usr/afs/bin/pts membership admin -cell <cell name> -noauth
      Groups admin (id: 1) is a member of:
        system:administrators

Restart all AFS Server processes

Code Listing 4.13

# /usr/afs/bin/bos restart <server name> -all -cell <cell name> -noauth

Starting the File Server, Volume Server and Salvager

Start the fs process, which consists of the File Server, Volume Server and Salvager (fileserver, volserver and salvager processes).

Code Listing 4.14

# /usr/afs/bin/bos create <server name> fs fs /usr/afs/bin/fileserver 
		                              /usr/afs/bin/volserver 
        		                      /usr/afs/bin/salvager 
                		              -cell <cell name> -noauth

Verify that all processes are running

Code Listing 4.15

 # /usr/afs/bin/bos status <server name> -long -noauth
 Instance kaserver, (type is simple) currently running normally.
     Process last started at Mon Jun  4 21:07:17 2001 (2 proc starts)
     Last exit at Mon Jun  4 21:07:17 2001
     Command 1 is '/usr/afs/bin/kaserver'

 Instance buserver, (type is simple) currently running normally.
     Process last started at Mon Jun  4 21:07:17 2001 (2 proc starts)
     Last exit at Mon Jun  4 21:07:17 2001
     Command 1 is '/usr/afs/bin/buserver'

 Instance ptserver, (type is simple) currently running normally.
     Process last started at Mon Jun  4 21:07:17 2001 (2 proc starts)
     Last exit at Mon Jun  4 21:07:17 2001
     Command 1 is '/usr/afs/bin/ptserver'

 Instance vlserver, (type is simple) currently running normally.
     Process last started at Mon Jun  4 21:07:17 2001 (2 proc starts)
     Last exit at Mon Jun  4 21:07:17 2001
     Command 1 is '/usr/afs/bin/vlserver'

 Instance fs, (type is fs) currently running normally.
     Auxiliary status is: file server running.
     Process last started at Mon Jun  4 21:09:30 2001 (2 proc starts)
     Command 1 is '/usr/afs/bin/fileserver'
     Command 2 is '/usr/afs/bin/volserver'
     Command 3 is '/usr/afs/bin/salvager'

Your next action depends on whether you have ever run AFS file server machines in the cell:

If you are installing the first AFS Server ever in the cell create the first AFS volume, root.afs

Note: For the partition name argument, substitute the name of one of the machine's AFS Server partitions. By convention these partitions are named /vicepx, where x is in the range of a-z.

Code Listing 4.16

  # /usr/afs/bin/vos create <server name> 
                            <partition name> root.afs 
				-cell <cell name> -noauth

If there are existing AFS file server machines and volumes in the cell issue the vos sncvldb and vos syncserv commands to synchronize the VLDB (Volume Location Database) with the actual state of volumes on the local machine. This will copy all necessary data to your new server.

If the command fails with the message "partition /vicepa does not exist on the server", ensure that the partition is mounted before running OpenAFS servers, or mount the directory and restart the processes using /usr/afs/bin/bos restart <server name> -all -cell <cell name> -noauth.

Code Listing 4.17

  # /usr/afs/bin/vos syncvldb <server name> -cell <cell name> -verbose -noauth
  # /usr/afs/bin/vos syncserv <server name> -cell <cell name> -verbose -noauth

Starting the Server Portion of the Update Server

Code Listing 4.18

# /usr/afs/bin/bos create <server name> 
                    upserver simple "/usr/afs/bin/upserver 
		    -crypt /usr/afs/etc -clear /usr/afs/bin" 
		    -cell <cell name> -noauth

Configuring the Top Level of the AFS filespace

First you need to set some acl's, so that any user can lookup /afs.

Code Listing 4.19

# /usr/afs/bin/fs setacl /afs system:anyuser rl

Then you need to create the root volume, mount it readonly on /afs/<cell name> and read/write on /afs/.<cell name>

Code Listing 4.20

# /usr/afs/bin/vos create <server name><partition name> root.cell
# /usr/afs/bin/fs mkmount /afs/<cell name> root.cell 
# /usr/afs/bin/fs setacl /afs/<cell name> system:anyuser rl
# /usr/afs/bin/fs mkmount /afs/.<cell name> root.cell -rw

Finally you're done !!! You should now have a working AFS file server on your local network. Time to get a big cup of coffee and print out the AFS documentation !!!

Note: It is very important for the AFS server to function properly, that all system clock's are synchronized. This is best accomplished by installing a ntp server on one machine (e.g. the AFS server) and synchronize all client clock's with the ntp client. This can also be done by the afs client.

5. Basic Administration

Disclaimer

OpenAFS is an extensive technology. Please read the AFS documentation for more information. We only list a few administrative tasks in this chapter.

Configuring PAM to Acquire an AFS Token on Login

To use AFS you need to authenticate against the KA Server if using an implementation AFS Kerberos 4, or against a Kerberos 5 KDC if using MIT, Heimdal, or ShiShi Kerberos 5. However in order to login to a machine you will also need a user account, this can be local in /etc/passwd, NIS, LDAP (OpenLDAP), or a Hesiod database. PAM allows Gentoo to tie the authentication against AFS and login to the user account.

You will need to update /etc/pam.d/system-auth which is used by the other configurations. "use_first_pass" indicates it will be checked first against the user login, and "ignore_root" stops the local super user being checked so as to order to allow login if AFS or the network fails.

Code Listing 5.1: /etc/pam.d/system-auth

auth       required     /lib/security/pam_env.so
auth       sufficient   /lib/security/pam_unix.so likeauth nullok
auth       sufficient   /usr/afsws/lib/pam_afs.so.1 use_first_pass ignore_root
auth       required     /lib/security/pam_deny.so

account    required     /lib/security/pam_unix.so

password   required     /lib/security/pam_cracklib.so retry=3
password   sufficient   /lib/security/pam_unix.so nullok md5 shadow use_authtok
password   required     /lib/security/pam_deny.so

session    required     /lib/security/pam_limits.so
session    required     /lib/security/pam_unix.so

In order for sudo to keep the real user's token and to prevent local users gaining AFS access change /etc/pam.d/su as follows:

Code Listing 5.2: /etc/pam.d/su

# Here, users with uid > 100 are considered to belong to AFS and users with
# uid <= 100 are ignored by pam_afs.
auth       sufficient   /usr/afsws/lib/pam_afs.so.1 ignore_uid 100

auth       sufficient   /lib/security/pam_rootok.so

# If you want to restrict users begin allowed to su even more,
# create /etc/security/suauth.allow (or to that matter) that is only
# writable by root, and add users that are allowed to su to that
# file, one per line.
#auth       required     /lib/security/pam_listfile.so item=ruser \
#       sense=allow onerr=fail file=/etc/security/suauth.allow

# Uncomment this to allow users in the wheel group to su without
# entering a passwd.
#auth       sufficient   /lib/security/pam_wheel.so use_uid trust

# Alternatively to above, you can implement a list of users that do
# not need to supply a passwd with a list.
#auth       sufficient   /lib/security/pam_listfile.so item=ruser \
#       sense=allow onerr=fail file=/etc/security/suauth.nopass

# Comment this to allow any user, even those not in the 'wheel'
# group to su
auth       required     /lib/security/pam_wheel.so use_uid

auth       required     /lib/security/pam_stack.so service=system-auth

account    required     /lib/security/pam_stack.so service=system-auth

password   required     /lib/security/pam_stack.so service=system-auth

session    required     /lib/security/pam_stack.so service=system-auth
session    optional     /lib/security/pam_xauth.so

# Here we prevent the real user id's token from being dropped
session    optional     /usr/afsws/lib/pam_afs.so.1 no_unlog

The contents of this document are licensed under the Creative Commons - Attribution / Share Alike license.

Updated September 22, 2004

Holger Brueckner
Editor

Benny Chuang
Editor

Tiemo Kieft
Editor

Steven McCoy
Editor

Summary: This guide shows you how to install a openafs server and client on gentoo linux