Package Building Procedures

The FreeBSD Ports Management Team

$FreeBSD: doc/en_US.ISO8859-1/articles/portbuild/article.sgml,v 1.67 2011/01/23 04:07:34 linimon Exp $


Table of Contents
1 Introduction
2 Build Client Management
3 Chroot Build Environment Setup
4 Starting the Build
5 Anatomy of a Build
6 Build Maintenance
7 Monitoring the Build
8 Dealing With Build Errors
9 Release Builds
10 Uploading Packages
11 Experimental Patches Builds
12 How to configure a new package building node
13 How to configure a new FreeBSD branch
14 How to configure a new architecture
15 How to configure a new head node (pointyhat instance)
16 Procedures for dealing with disk failures

1 Introduction

In order to provide pre-compiled binaries of third-party applications for FreeBSD, the Ports Collection is regularly built on one of the “Package Building Clusters.” Currently, the main cluster in use is at http://pointyhat.FreeBSD.org.

This article documents the internal workings of the cluster.

Note: Many of the details will be of interest only to those on the Ports Management team.

1.1 The codebase

Most of the package building magic occurs under the /var/portbuild directory. Unless otherwise specified, all paths will be relative to this location. ${arch} will be used to specify one of the package architectures (amd64, i386™, ia64, powerpc, and SPARC64®), and ${branch} will be used to specify the build branch (7, 7-exp, 8, 8-exp, 9, 9-exp).

Note: Packages are no longer built for Releases 4, 5, or 6, nor for the alpha architecture.

The scripts that control all of this live in /var/portbuild/scripts/. These are the checked-out copies from /usr/ports/Tools/portbuild/scripts/.

Typically, incremental builds are done that use previous packages as dependencies; this takes less time, and puts less load on the mirrors. Full builds are usually only done:

  • right after release time, for the -STABLE branches

  • periodically to test changes to -CURRENT

  • for experimental builds

1.2 Notes on the codebase

Until mid-2010, the scripts were completely specific to pointyhat as the head (dispatch) node. During the summer of 2010, a significant rewrite was done in order to allow for other hosts to be head nodes. Among the changes were:

  • removal of the hard-coding of the string pointyhat

  • factoring out all configuration constants (which were previously scattered throughout the code) into configuration files (see below)

  • appending the hostname to the directories specified by buildid (this will allow directories to be unambigious when copied between machines.)

  • making the scripts more robust in terms of setting up directories and symlinks

  • where necessary, changing certain script invocations to make all the above easier

This document was originally written before these changes were made. Where things such as script invocations have changed, they are denoted as new codebase: as opposed to old codebase:.

Note: As of December 2010, pointyhat is still running on the old codebase, until the new codebase is considered rock-solid.