6.5. Building Mechanisms

6.5.1. Building Ports in Parallel

The FreeBSD ports framework supports parallel building using multiple make sub-processes, which allows SMP systems to utilize all of their available CPU power, allowing port builds to be faster and more effective.

This is achieved by passing -jX flag to make(1) running on vendor code. This is the default build behavior of ports. Unfortunately, not all ports handle parallel building well and it may be required to explicitly disable this feature by adding the MAKE_JOBS_UNSAFE=yes variable. It is used when a port is known to be broken with -jX due to race conditions causing intermittent build failures.

Important:

When setting MAKE_JOBS_UNSAFE, it is very important to explain either with a comment in the Makefile, or at least in the commit message, why the port does not build when enabling. Otherwise, it is almost impossible to either fix the problem, or test if it has been fixed when committing an update at a later date.

6.5.2. make, gmake, and imake

Several differing make implementations exist. Ported software often requires a particular implementation, like GNU make, known in FreeBSD as gmake.

If the port uses GNU make, add gmake to USES.

MAKE_CMD can be used to reference the specific command configured by the USES setting in the port's Makefile. Only use MAKE_CMD within the application Makefiles in WRKSRC to call the make implementation expected by the ported software.

If the port is an X application that uses imake to create Makefiles from Imakefiles, set USES= imake.. See the USES=imake section of Chapter 17, Using USES Macros for more details.

If the port's source Makefile has something other than all as the main build target, set ALL_TARGET accordingly. The same goes for install and INSTALL_TARGET.

6.5.3. configure Script

If the port uses the configure script to generate Makefile from Makefile.in, set GNU_CONFIGURE=yes. To give extra arguments to the configure script (the default argument is --prefix=${PREFIX} --infodir=${PREFIX}/${INFO_PATH} --mandir=${MANPREFIX}/man --build=${CONFIGURE_TARGET}), set those extra arguments in CONFIGURE_ARGS. Extra environment variables can be passed using CONFIGURE_ENV.

Table 6.1. Variables for Ports That Use configure
VariableMeans
GNU_CONFIGUREThe port uses configure script to prepare build.
HAS_CONFIGURESame as GNU_CONFIGURE, except default configure target is not added to CONFIGURE_ARGS.
CONFIGURE_ARGSAdditional arguments passed to configure script.
CONFIGURE_ENVAdditional environment variables to be set for configure script run.
CONFIGURE_TARGETOverride default configure target. Default value is ${MACHINE_ARCH}-portbld-freebsd${OSREL}.

6.5.4. Using cmake

For ports that use CMake, define USES= cmake.

Table 6.2. Variables for Ports That Use cmake
VariableMeans
CMAKE_ARGSPort specific CMake flags to be passed to the cmake binary.
CMAKE_ONFor each entry in CMAKE_ON, an enabled boolean value is added to CMAKE_ARGS. See Example 6.3, “CMAKE_ON and CMAKE_OFF.
CMAKE_OFFFor each entry in CMAKE_OFF, a disabled boolean value is added to CMAKE_ARGS. See Example 6.3, “CMAKE_ON and CMAKE_OFF.
CMAKE_BUILD_TYPEType of build (CMake predefined build profiles). Default is Release, or Debug if WITH_DEBUG is set.
CMAKE_SOURCE_PATHPath to the source directory. Default is ${WRKSRC}.
CONFIGURE_ENVAdditional environment variables to be set for the cmake binary.

Table 6.3. Variables the Users Can Define for cmake Builds
VariableMeans
CMAKE_NOCOLORDisables color build output. Default not set, unless BATCH or PACKAGE_BUILDING are set.

CMake supports these build profiles: Debug, Release, RelWithDebInfo and MinSizeRel. Debug and Release profiles respect system *FLAGS, RelWithDebInfo and MinSizeRel will set CFLAGS to -O2 -g and -Os -DNDEBUG correspondingly. The lower-cased value of CMAKE_BUILD_TYPE is exported to PLIST_SUB and must be used if the port installs *.cmake depending on the build type (see devel/kf5-kcrash for an example). Please note that some projects may define their own build profiles and/or force particular build type by setting CMAKE_BUILD_TYPE in CMakeLists.txt. To make a port for such a project respect CFLAGS and WITH_DEBUG, the CMAKE_BUILD_TYPE definitions must be removed from those files.

Most CMake-based projects support an out-of-source method of building. The out-of-source build for a port is the default setting. An in-source build can be requested by using the :insource suffix. With out-of-source builds, CONFIGURE_WRKSRC, BUILD_WRKSRC and INSTALL_WRKSRC will be set to ${WRKDIR}/.build and this directory will be used to keep all files generated during configuration and build stages, leaving the source directory intact.

Example 6.2. USES= cmake Example

This snippet demonstrates the use of CMake for a port. CMAKE_SOURCE_PATH is not usually required, but can be set when the sources are not located in the top directory, or if only a subset of the project is intended to be built by the port.

USES=			cmake
CMAKE_SOURCE_PATH=	${WRKSRC}/subproject

Example 6.3. CMAKE_ON and CMAKE_OFF

When adding boolean values to CMAKE_ARGS, it is easier to use the CMAKE_ON and CMAKE_OFF variables instead. This:

CMAKE_ON=	VAR1 VAR2
CMAKE_OFF=	VAR3

Is equivalent to:

CMAKE_ARGS=	-DVAR1:BOOL=TRUE -DVAR2:BOOL=TRUE -DVAR3:BOOL=FALSE

Important:

This is only for the default values off CMAKE_ARGS. The helpers described in Section 5.13.3.4.2, “OPT_CMAKE_BOOL and OPT_CMAKE_BOOL_OFF use the same semantics, but for optional values.


6.5.5. Using scons

If the port uses SCons, define USES=scons.

To make third party SConstruct respect everything that is passed to SCons in the environment (that is, most importantly, CC/CXX/CFLAGS/CXXFLAGS), patch SConstruct so build Environment is constructed like this:

env = Environment(**ARGUMENTS)

It may be then modified with env.Append and env.Replace.

6.5.6. Building Rust Applications with cargo

For ports that use Cargo, define USES=cargo.

Table 6.4. Variables the Users Can Define for cargo Builds
VariableDefaultDescription
CARGO_CRATES List of crates the port depends on. Each entry needs to have a format like cratename-semver for example, libc-0.2.40. Port maintainers can generate this list from Cargo.lock using make cargo-crates. Manually bumping crate versions is possible but be mindful of transitive dependencies.
CARGO_FEATURES List of application features to build (space separated list). To deactivate all default features add the special token --no-default-features to CARGO_FEATURES. Manually passing it to CARGO_BUILD_ARGS, CARGO_INSTALL_ARGS, and CARGO_TEST_ARGS is not needed.
CARGO_CARGOTOML${WRKSRC}/Cargo.tomlThe path to the Cargo.toml to use.
CARGO_CARGOLOCK${WRKSRC}/Cargo.lockThe path to the Cargo.lock to use for make cargo-crates. It is possible to specify more than one lock file when necessary.
CARGO_ENV A list of environment variables to pass to Cargo similar to MAKE_ENV.
RUSTFLAGS Flags to pass to the Rust compiler.
CARGO_CONFIGUREyesUse the default do-configure.
CARGO_UPDATE_ARGS Extra arguments to pass to Cargo during the configure phase. Valid arguments can be looked up with cargo update --help.
CARGO_BUILDDEPyesAdd a build dependency on lang/rust.
CARGO_CARGO_BIN${LOCALBASE}/bin/cargoLocation of the cargo binary.
CARGO_BUILDyesUse the default do-build.
CARGO_BUILD_ARGS Extra arguments to pass to Cargo during the build phase. Valid arguments can be looked up with cargo build --help.
CARGO_INSTALLyesUse the default do-install.
CARGO_INSTALL_ARGS Extra arguments to pass to Cargo during the install phase. Valid arguments can be looked up with cargo install --help.
CARGO_INSTALL_PATH.Path to the crate to install. This is passed to cargo install via its --path argument. When multiple paths are specified cargo install is run multiple times.
CARGO_TESTyesUse the default do-test.
CARGO_TEST_ARGS Extra arguments to pass to Cargo during the test phase. Valid arguments can be looked up with cargo test --help.
CARGO_TARGET_DIR${WRKDIR}/targetLocation of the cargo output directory.
CARGO_DIST_SUBDIRrust/cratesDirectory relative to DISTDIR where the crate distribution files will be stored.
CARGO_VENDOR_DIR${WRKSRC}/cargo-cratesLocation of the vendor directory where all crates will be extracted to. Try to keep this under PATCH_WRKSRC, so that patches can be applied easily.
CARGO_USE_GITHUBnoEnable fetching of crates locked to specific Git commits on GitHub via GH_TUPLE. This will try to patch all Cargo.toml under WRKDIR to point to the offline sources instead of fetching them from a Git repository during the build.
CARGO_USE_GITLABnoSame as CARGO_USE_GITHUB but for GitLab instances and GL_TUPLE.

Example 6.4. Creating a Port for a Simple Rust Application

Creating a Cargo based port is a three stage process. First we need to provide a ports template that fetches the application distribution file:

PORTNAME=	tokei
DISTVERSIONPREFIX=	v
DISTVERSION=	7.0.2
CATEGORIES=	devel

MAINTAINER=	tobik@FreeBSD.org
COMMENT=	Display statistics about your code

USES=		cargo
USE_GITHUB=	yes
GH_ACCOUNT=	Aaronepower

.include <bsd.port.mk>

Generate an initial distinfo:

% make makesum
=> Aaronepower-tokei-v7.0.2_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/Aaronepower/tokei/tar.gz/v7.0.2?dummy=/Aaronepower-tokei-v7.0.2_GH0.tar.gz
fetch: https://codeload.github.com/Aaronepower/tokei/tar.gz/v7.0.2?dummy=/Aaronepower-tokei-v7.0.2_GH0.tar.gz: size of remote file is not known
Aaronepower-tokei-v7.0.2_GH0.tar.gz                     45 kB  239 kBps 00m00s

Now the distribution file is ready to use and we can go ahead and extract crate dependencies from the bundled Cargo.lock:

% make cargo-crates
CARGO_CRATES=   aho-corasick-0.6.4 \
                ansi_term-0.11.0 \
                arrayvec-0.4.7 \
                atty-0.2.9 \
                bitflags-1.0.1 \
                byteorder-1.2.2 \
                [...]

The output of this command needs to be pasted directly into the Makefile:

PORTNAME=	tokei
DISTVERSIONPREFIX=	v
DISTVERSION=	7.0.2
CATEGORIES=	devel

MAINTAINER=	tobik@FreeBSD.org
COMMENT=	Display statistics about your code

USES=		cargo
USE_GITHUB=	yes
GH_ACCOUNT=	Aaronepower

CARGO_CRATES=   aho-corasick-0.6.4 \
                ansi_term-0.11.0 \
                arrayvec-0.4.7 \
                atty-0.2.9 \
                bitflags-1.0.1 \
                byteorder-1.2.2 \
                [...]

.include <bsd.port.mk>

distinfo needs to be regenerated to contain all the crate distribution files:

% make makesum
=> rust/crates/aho-corasick-0.6.4.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://crates.io/api/v1/crates/aho-corasick/0.6.4/download?dummy=/rust/crates/aho-corasick-0.6.4.tar.gz
rust/crates/aho-corasick-0.6.4.tar.gz         100% of   24 kB 6139 kBps 00m00s
=> rust/crates/ansi_term-0.11.0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://crates.io/api/v1/crates/ansi_term/0.11.0/download?dummy=/rust/crates/ansi_term-0.11.0.tar.gz
rust/crates/ansi_term-0.11.0.tar.gz           100% of   16 kB   21 MBps 00m00s
=> rust/crates/arrayvec-0.4.7.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://crates.io/api/v1/crates/arrayvec/0.4.7/download?dummy=/rust/crates/arrayvec-0.4.7.tar.gz
rust/crates/arrayvec-0.4.7.tar.gz             100% of   22 kB 3237 kBps 00m00s
=> rust/crates/atty-0.2.9.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://crates.io/api/v1/crates/atty/0.2.9/download?dummy=/rust/crates/atty-0.2.9.tar.gz
rust/crates/atty-0.2.9.tar.gz                 100% of 5898  B   81 MBps 00m00s
=> rust/crates/bitflags-1.0.1.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
[...]

The port is now ready for a test build and further adjustments like creating a plist, writing a description, adding license information, options, etc. as normal.

If you are not testing your port in a clean environment like with Poudriere, remember to run make clean before any testing.


Example 6.5. Enabling Additional Application Features

Some applications define additional features in their Cargo.toml. They can be compiled in by setting CARGO_FEATURES in the port.

Here we enable Tokei's json and yaml features:

CARGO_FEATURES=	json yaml

Example 6.6. Encoding Application Features As Port Options

An example [features] section in Cargo.toml could look like this:

[features]
pulseaudio_backend = ["librespot-playback/pulseaudio-backend"]
portaudio_backend = ["librespot-playback/portaudio-backend"]
default = ["pulseaudio_backend"]

pulseaudio_backend is a default feature. It is always enabled unless we explicitly turn off default features by adding --no-default-features to CARGO_FEATURES. Here we turn the portaudio_backend and pulseaudio_backend features into port options:

CARGO_FEATURES=	--no-default-features

OPTIONS_DEFINE=	PORTAUDIO PULSEAUDIO

PORTAUDIO_VARS=		CARGO_FEATURES+=portaudio_backend
PULSEAUDIO_VARS=	CARGO_FEATURES+=pulseaudio_backend

Example 6.7. Listing Crate Licenses

Crates have their own licenses. It is important to know what they are when adding a LICENSE block to the port (see Section 5.7, “Licenses”). The helper target cargo-crates-licenses will try to list all the licenses of all crates defined in CARGO_CRATES.

% make cargo-crates-licenses
aho-corasick-0.6.4  Unlicense/MIT
ansi_term-0.11.0    MIT
arrayvec-0.4.7      MIT/Apache-2.0
atty-0.2.9          MIT
bitflags-1.0.1      MIT/Apache-2.0
byteorder-1.2.2     Unlicense/MIT
[...]

Note:

The license names make cargo-crates-licenses outputs are SPDX 2.1 licenses expression which do not match the license names defined in the ports framework. They need to be translated to the names from Table 5.7, “Predefined License List”.


6.5.7. Using meson

For ports that use Meson, define USES=meson.

Table 6.5. Variables for Ports That Use meson
VariableDescription
MESON_ARGSPort specific Meson flags to be passed to the meson binary.
MESON_BUILD_DIRPath to the build directory relative to WRKSRC. Default is _build.

Example 6.8. USES=meson Example

This snippet demonstrates the use of Meson for a port.

USES=		meson
MESON_ARGS=	-Dfoo=enabled

6.5.8. Building Go Applications

For ports that use Go, define USES=go. Refer to Section 17.35, “go for a list of variables that can be set to control the build process.

Example 6.9. Creating a Port for a Go Modules Based Application

Creating a Go based port is a five stage process. First we need to provide a ports template that fetches the application distribution file:

PORTNAME=	ghq
DISTVERSIONPREFIX=	v
DISTVERSION=	0.12.5
CATEGORIES=	devel

MAINTAINER=	tobik@FreeBSD.org
COMMENT=	Remote repository management made easy

USES=		go:modules
USE_GITHUB=	yes
GH_ACCOUNT=	motemen

.include <bsd.port.mk>

Generate an initial distinfo:

% make makesum
===>  License MIT accepted by the user
=> motemen-ghq-v0.12.5_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/motemen/ghq/tar.gz/v0.12.5?dummy=/motemen-ghq-v0.12.5_GH0.tar.gz
fetch: https://codeload.github.com/motemen/ghq/tar.gz/v0.12.5?dummy=/motemen-ghq-v0.12.5_GH0.tar.gz: size of remote file is not known
motemen-ghq-v0.12.5_GH0.tar.gz                          32 kB  177 kBps    00s

Now the distribution file is ready to use and we can extract the required Go module dependencies. This step requires having ports-mgmt/modules2tuple installed:

% make gomod-vendor
[...]
GH_TUPLE=	\
		Songmu:gitconfig:v0.0.2:songmu_gitconfig/vendor/github.com/Songmu/gitconfig \
		daviddengcn:go-colortext:186a3d44e920:daviddengcn_go_colortext/vendor/github.com/daviddengcn/go-colortext \
		go-yaml:yaml:v2.2.2:go_yaml_yaml/vendor/gopkg.in/yaml.v2 \
		golang:net:3ec191127204:golang_net/vendor/golang.org/x/net \
		golang:sync:112230192c58:golang_sync/vendor/golang.org/x/sync \
		golang:xerrors:3ee3066db522:golang_xerrors/vendor/golang.org/x/xerrors \
		motemen:go-colorine:45d19169413a:motemen_go_colorine/vendor/github.com/motemen/go-colorine \
		urfave:cli:v1.20.0:urfave_cli/vendor/github.com/urfave/cli

The output of this command needs to be pasted directly into the Makefile:

PORTNAME=	ghq
DISTVERSIONPREFIX=	v
DISTVERSION=	0.12.5
CATEGORIES=	devel

MAINTAINER=	tobik@FreeBSD.org
COMMENT=	Remote repository management made easy

USES=		go:modules
USE_GITHUB=	yes
GH_ACCOUNT=	motemen
GH_TUPLE=	Songmu:gitconfig:v0.0.2:songmu_gitconfig/vendor/github.com/Songmu/gitconfig \
		daviddengcn:go-colortext:186a3d44e920:daviddengcn_go_colortext/vendor/github.com/daviddengcn/go-colortext \
		go-yaml:yaml:v2.2.2:go_yaml_yaml/vendor/gopkg.in/yaml.v2 \
		golang:net:3ec191127204:golang_net/vendor/golang.org/x/net \
		golang:sync:112230192c58:golang_sync/vendor/golang.org/x/sync \
		golang:xerrors:3ee3066db522:golang_xerrors/vendor/golang.org/x/xerrors \
		motemen:go-colorine:45d19169413a:motemen_go_colorine/vendor/github.com/motemen/go-colorine \
		urfave:cli:v1.20.0:urfave_cli/vendor/github.com/urfave/cli

.include <bsd.port.mk>

distinfo needs to be regenerated to contain all the distribution files:

% make makesum
=> Songmu-gitconfig-v0.0.2_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/Songmu/gitconfig/tar.gz/v0.0.2?dummy=/Songmu-gitconfig-v0.0.2_GH0.tar.gz
fetch: https://codeload.github.com/Songmu/gitconfig/tar.gz/v0.0.2?dummy=/Songmu-gitconfig-v0.0.2_GH0.tar.gz: size of remote file is not known
Songmu-gitconfig-v0.0.2_GH0.tar.gz                    5662  B  936 kBps    00s
=> daviddengcn-go-colortext-186a3d44e920_GH0.tar.gz doesn't seem to exist in /usr/ports/distfiles/.
=> Attempting to fetch https://codeload.github.com/daviddengcn/go-colortext/tar.gz/186a3d44e920?dummy=/daviddengcn-go-colortext-186a3d44e920_GH0.tar.gz
fetch: https://codeload.github.com/daviddengcn/go-colortext/tar.gz/186a3d44e920?dummy=/daviddengcn-go-colortext-186a3d44e920_GH0.tar.gz: size of remote file is not known
daviddengcn-go-colortext-186a3d44e920_GH0.tar.        4534  B 1098 kBps    00s
[...]

The port is now ready for a test build and further adjustments like creating a plist, writing a description, adding license information, options, etc. as normal.

If you are not testing your port in a clean environment like with Poudriere, remember to run make clean before any testing.


Example 6.10. Setting Output Binary Name or Installation Path

Some ports need to install the resulting binary under a different name or to a path other than the default ${PREFIX}/bin. This can be done by using GO_TARGET tuple syntax, for example:

GO_TARGET=  ./cmd/ipfs:ipfs-go

will install ipfs binary as ${PREFIX}/bin/ipfs-go and

GO_TARGET=  ./dnscrypt-proxy:${PREFIX}/sbin/dnscrypt-proxy

will install dnscrypt-proxy to ${PREFIX}/sbin.


6.5.9. Building Haskell Applications with cabal

For ports that use Cabal, build system defines USES=cabal. Refer to Section 17.8, “cabal for a list of variables that can be set to control the build process.

Example 6.11. Creating a Port for a Hackage-hosted Haskell Application

When preparing a Haskell Cabal port, the devel/hs-cabal-install program is required, so make sure it is installed beforehand. First we need to define common ports variables that allows cabal-install to fetch the package distribution file:

PORTNAME=	ShellCheck
DISTVERSION=	0.6.0
CATEGORIES=	devel

MAINTAINER=	haskell@FreeBSD.org
COMMENT=	Shell script analysis tool

USES=		cabal

.include <bsd.port.mk>

This minimal Makefile allows us to fetch the distribution file:

% make cabal-extract
[...]
Downloading the latest package list from hackage.haskell.org
cabal get ShellCheck-0.6.0
Downloading  ShellCheck-0.6.0
Downloaded   ShellCheck-0.6.0
Unpacking to ShellCheck-0.6.0/

Now we have ShellCheck.cabal package description file, which allows us to fetch all package's dependencies, including transitive ones:

% make cabal-extract-deps
[...]
Resolving dependencies...
Downloading  base-orphans-0.8.2
Downloaded   base-orphans-0.8.2
Downloading  primitive-0.7.0.0
Starting     base-orphans-0.8.2 (lib)
Building     base-orphans-0.8.2 (lib)
Downloaded   primitive-0.7.0.0
Downloading  dlist-0.8.0.7
[...]

As a side effect, the package's dependencies are also compiled, so the command may take some time. Once done, a list of required dependencies can generated:

% make make-use-cabal
USE_CABAL=QuickCheck-2.12.6.1 \
hashable-1.3.0.0 \
integer-logarithms-1.0.3 \
[...]

Haskell packages may contain revisions, just like FreeBSD ports. Revisions can affect only .cabal files, but it is still important to pull them in. To check USE_CABAL items for available revision updates, run following command:

% make make-use-cabal-revs
USE_CABAL=QuickCheck-2.12.6.1_1 \
hashable-1.3.0.0 \
integer-logarithms-1.0.3_2 \
[...]

Note additional version numbers after _ symbol. Put newly generated USE_CABAL list instead of an old one.

Finally, distinfo needs to be regenerated to contain all the distribution files:

% make makesum
=> ShellCheck-0.6.0.tar.gz doesn't seem to exist in /usr/local/poudriere/ports/git/distfiles/cabal.
=> Attempting to fetch https://hackage.haskell.org/package/ShellCheck-0.6.0/ShellCheck-0.6.0.tar.gz
ShellCheck-0.6.0.tar.gz                                136 kB  642 kBps    00s
=> QuickCheck-2.12.6.1/QuickCheck-2.12.6.1.tar.gz doesn't seem to exist in /usr/local/poudriere/ports/git/distfiles/cabal.
=> Attempting to fetch https://hackage.haskell.org/package/QuickCheck-2.12.6.1/QuickCheck-2.12.6.1.tar.gz
QuickCheck-2.12.6.1/QuickCheck-2.12.6.1.tar.gz          65 kB  361 kBps    00s
[...]

The port is now ready for a test build and further adjustments like creating a plist, writing a description, adding license information, options, etc. as normal.

If you are not testing your port in a clean environment like with Poudriere, remember to run make clean before any testing.


All FreeBSD documents are available for download at https://download.freebsd.org/ftp/doc/

Questions that are not answered by the documentation may be sent to <freebsd-questions@FreeBSD.org>.
Send questions about this document to <freebsd-doc@FreeBSD.org>.