Package Building Procedures
Prev Next

6 Build Maintenance

There are several cases where you will need to manually clean up a build:

  1. You have manually interrupted it.

  2. pointyhat has been rebooted while a build was running.

  3. qmanager has crashed and has been restarted.

6.1 Interrupting a Build

Manually interrupting a build is a bit messy. First you need to identify the tty in which it's running (either record the output of tty(1) when you start the build, or use ps x to identify it. You need to make sure that nothing else important is running in this tty, e.g. ps -t p1 or whatever. If there is not, you can just kill off the whole term easily with pkill -t pts/1; otherwise issue a kill -HUP in there by, for example, ps -t pts/1 -o pid= | xargs kill -HUP. Replace p1 by whatever the tty is, of course.

The package builds dispatched by make to the client machines will clean themselves up after a few minutes (check with ps x until they all go away).

If you do not kill make(1), then it will spawn more jobs. If you do not kill dopackages, then it will restart the entire build. If you do not kill the pdispatch processes, they'll keep going (or respawn) until they've built their package.

6.2 Cleaning up a Build

To free up resources, you will need to clean up client machines by running build cleanup command. For example:

% /var/portbuild/scripts/build cleanup i386 8-exp 20080714120411 -full


If you forget to do this, then the old build chroots will not be cleaned up for 24 hours, and no new jobs will be dispatched in their place since pointyhat thinks the job slot is still occupied.

To check, cat ~/loads/* to display the status of client machines; the first column is the number of jobs it thinks is running, and this should be roughly concordant with the load average. loads is refreshed every 2 minutes. If you do ps x | grep pdispatch and it's less than the number of jobs that loads thinks are in use, you're in trouble.

You may have problem with the umount commands hanging. If so, you are going to have to use the allgohans script to run an ssh(1) command across all clients for that buildenv. For example:

ssh -l root gohan24 df
will get you a df, and 
allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports"
allgohans "umount -f pointyhat.freebsd.org:/var/portbuild/i386/8-exp/src"
are supposed to get rid of the hanging mounts. You will have to keep doing them since there can be multiple mounts.

Note: Ignore the following:

umount: pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports: statfs: No such file or directory
umount: pointyhat.freebsd.org:/var/portbuild/i386/8-exp/ports: unknown file system
umount: Cleanup of /x/tmp/8-exp/chroot/53837/compat/linux/proc failed!
/x/tmp/8-exp/chroot/53837/compat/linux/proc: not a file system root directory
The former 2 mean that that client did not have those mounted; the latter 2 are a bug.

You may also see messages about procfs.

After you have done all the above, remove the ${arch}/lock file before trying to restart the build. If you do not, dopackages will simply exit.

If you have to do a ports tree update before restarting, you may have to rebuild either duds, INDEX, or both.

6.3 Maintaining builds with the build command

Here are the rest of the options for the build command:

Prev Home Next
Anatomy of a Build   Monitoring the Build