7. Building FreeBSD Installation Media

7. Building FreeBSD Installation Media
Prev		Next

This section describes the general procedures producing FreeBSD development snapshots and releases.

7.1. Release Build Scripts

This section describes the build scripts used by FreeBSD Release Engineering Team to produce development snapshots and releases.

7.1.1. The `release.sh` Script

Prior to FreeBSD 9.0-RELEASE, src/release/Makefile was updated to support bsdinstall(8), and the src/release/generate-release.sh script was introduced as a wrapper to automate invoking the release(7) targets.

Prior to FreeBSD 9.2-RELEASE, src/release/release.sh was introduced, which heavily based on src/release/generate-release.sh included support to specify configuration files to override various options and environment variables. Support for configuration files provided support for cross building each architecture for a release by specifying a separate configuration file for each invocation.

As a brief example of using src/release/release.sh to build a single release in /scratch:

# /bin/sh /usr/src/release/release.sh

As a brief example of using src/release/release.sh to build a single, cross-built release using a different target directory, create a custom release.conf containing:

# release.sh configuration for powerpc/powerpc64
CHROOTDIR="/scratch-powerpc64"
TARGET="powerpc"
TARGET_ARCH="powerpc64"
KERNEL="GENERIC64"

Then invoke src/release/release.sh as:

# /bin/sh /usr/src/release/release.sh -c $HOME/release.conf

See release(7) and src/release/release.conf.sample for more details and example usage.

7.1.2. The `thermite.sh` Wrapper Script

In order to make cross building the full set of architectures supported on a given branch faster, easier, and reduce human error factors, a wrapper script around src/release/release.sh was written to iterate through the various combinations of architectures and invoke src/release/release.sh using a configuration file specific to that architecture.

The wrapper script is called thermite.sh, which is available in the FreeBSD Subversion repository at svn://svn.freebsd.org/base/user/gjb/thermite/, in addition to configuration files used to build head/ and stable/12/ development snapshots.

Using thermite.sh is covered in Section 7.2, “Building FreeBSD Development Snapshots” and Section 7.3, “Building FreeBSD Releases”.

Each architecture and individual kernel have their own configuration file used by release.sh. Each branch has its own defaults-X.conf configuration which contains entries common throughout each architecture, where overrides or special variables are set and/or overridden in the per-build files.

The per-build configuration file naming scheme is in the form of ${revision}-${TARGET_ARCH}-${KERNCONF}-${type}.conf, where the uppercase variables are equivalent to what make(1) uses in the build system, and lowercase variables are set within the configuration files, mapping to the major version of the respective branch.

Each branch also has its own builds-X.conf configuration, which is used by thermite.sh. The thermite.sh script iterates through each ${revision}, ${TARGET_ARCH}, ${KERNCONF}, and ${type} value, creating a master list of what to build. However, a given combination from the list will only be built if the respective configuration file exists, which is where the naming scheme above is relevant.

There are two paths of file sourcing:

builds-12.conf -> main.conf
This controls thermite.sh behavior
12-amd64-GENERIC-snap.conf -> defaults-12.conf -> main.conf
This controls release/release.sh behavior within the build chroot(8)

Note:

The builds-12.conf, defaults-12.conf, and main.conf configuration files exist to reduce repetition between the various per-build files.

7.2. Building FreeBSD Development Snapshots

The official release build machines have a specific filesystem layout, which using ZFS, thermite.sh takes heavy advantage of with clones and snapshots, ensuring a pristine build environment.

The build scripts reside in /releng/scripts-snapshot/scripts or /releng/scripts-release/scripts respectively, to avoid collisions between an RC build from a releng branch versus a STABLE snapshot from the respective stable branch.

A separate dataset exists for the final build images, /snap/ftp. This directory contains both snapshots and releases directories. They are only used if the EVERYTHINGISFINE variable is defined in main.conf.

Note:

The EVERYTHINGISFINE variable name was chosen to avoid colliding with a variable that might be possibly set in the user environment, accidentally enabling the behavior that depends on it being defined.

As thermite.sh iterates through the master list of combinations and locates the per-build configuration file, a ZFS dataset is created under /releng, such as /releng/12-amd64-GENERIC-snap. The src/, ports/, and doc/ trees are checked out to separate ZFS datasets, such as /releng/12-src-snap, which are then cloned and mounted into the respective build datasets. This is done to avoid checking out a given tree more than once.

Assuming these filesystem paths, thermite.sh would be invoked as:

# cd /releng/scripts-snapshot/scripts
# ./setrev.sh -b stable/12/
# ./zfs-cleanup.sh -c ./builds-12.conf
# ./thermite.sh -c ./builds-12.conf

Once the builds have completed, additional helper scripts are available to generate development snapshot emails which are sent to the freebsd-snapshots@freebsd.org mailing list:

# cd /releng/scripts-snapshot/scripts
# ./get-checksums.sh -c ./builds-12.conf | ./generate-email.pl > snapshot-12-mail

Note:

The generated output should be double-checked for correctness, and the email itself should be PGP signed, in-line.

Note:

These helper scripts only apply to development snapshot builds. Announcements during the release cycle (excluding the final release announcement) are created from an email template. A sample of the email template currently used can be found here.

7.3. Building FreeBSD Releases

Similar to building FreeBSD development snapshots, thermite.sh would be invoked the same way. The difference between development snapshots and release builds, BETA and RC included, is that the chroot(8) configuration files must be named with release instead of snap as the "type", as mentioned above.

In addition, the BUILDTYPE and types must be changed from snap to release in defaults-12.conf and builds-12.conf, respectively.

When building BETA, RC, and the final RELEASE, also statically set BUILDSVNREV to the revision on the branch reflecting the name change, BUILDDATE to the date the builds are started in YYYYMMDD format. If the doc/ and ports/ trees have been tagged, also set PORTBRANCH and DOCBRANCH to the relevant tag path in the Subversion repository, replacing HEAD with the last changed revision. Also set releasesrc in builds-12.conf to the relevant branch, such as stable/12/ or releng/12.0/.

During the release cycle, a copy of CHECKSUM.SHA512 and CHECKSUM.SHA256 for each architecture are stored in the FreeBSD Release Engineering Team internal repository in addition to being included in the various announcement emails. Each MANIFEST containing the hashes of base.txz, kernel.txz, etc. are added to misc/freebsd-release-manifests in the Ports Collection, as well.

In preparation for the release build, several files need to be updated:

File to Edit	What to Change
`sys/conf/newvers.sh`	Update the `BRANCH` value to `RELEASE`
`UPDATING`	Add the anticipated announcement date
`lib/csu/common/crtbrand.c`	Replace `__FreeBSD_version` with the value in `sys/sys/param.h`

After building the final RELEASE, the releng/12.0/ branch is tagged as release/12.0.0/ using the revision from which the RELEASE was built. Similar to creating the stable/12/ and releng/12.0/ branches, this is done with svn cp. From the repository root:

% svn cp ^/releng/12.0/@r306420 release/12.0.0/
% svn commit release/12.0.0/

Prev		Next
6. Release from `stable/`	Home	8. Publishing FreeBSD Installation Media to Project Mirrors