NetBSD/doc/BUILDING.mdoc

.\"	$NetBSD: BUILDING.mdoc,v 1.94 2013/05/29 14:45:55 wiz Exp $
.\"
.\" Copyright (c) 2001-2011 The NetBSD Foundation, Inc.
.\" All rights reserved.
.\"
.\" This code is derived from software contributed to The NetBSD Foundation
.\" by Todd Vierling and Luke Mewburn.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\"    notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\"    notice, this list of conditions and the following disclaimer in the
.\"    documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
.\" ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
.\" TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
.\" PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
.\" BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
.\" CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
.\" SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
.\" CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
.\" ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
.\" POSSIBILITY OF SUCH DAMAGE.
.\"
.\" NOTE: After changing this file, run "make regen" in the src/doc
.\" directory, and check in both src/BUILDING and src/doc/BUILDING.mdoc.
.\"
.\" Toolchain prefix for commands
.ds toolprefix nb
.
.Dd November 8, 2012
.Dt BUILDING 8
.Os NetBSD
.
.Sh NAME
.
.Nm BUILDING
.Nd Procedure for building
.Nx
from source code.
.
.Sh REQUIREMENTS
.
.Nx
is designed to be buildable on most POSIX-compliant host systems.
The basic build procedure is the same whether compiling
.Em natively
(on the same
.Nx
architecture) or
.Em cross compiling
(on another architecture or OS).
.Pp
This source tree contains a special subtree,
.Dq tools ,
which uses the host system to create a build toolchain for the target
architecture.
The host system must have at least C and C++
compilers in order to create the toolchain
.Nm ( make
is not required); all other tools are created as part of the
.Nx
build process.
(See the environment variables section below if you need
to override or manually select your compilers.)
.
.Sh FILES
.
.Ss Source tree layout
.
.Bl -tag -width "BUILDING.mdoc"
.It Sy doc/BUILDING.mdoc
This document (in -mdoc troff format; the original copy).
.It Sy BUILDING
This document (in plaintext).
.It Sy tools/compat/README
Special notes for cross-hosting a NetBSD build on non-NetBSD platforms.
.It Sy Makefile
The main Makefile for
.Nx ;
should only be run for native builds with an appropriately up-to-date
version of
.Nx
.Xr make 1 .
(For building from out-of-date systems or on a non-native host, see the
.Nm build.sh
shell script.)
.It Sy UPDATING
Special notes for updating from an earlier revision of
.Nx .
It is important to read this file before every build of an updated
source tree.
.It Sy build.sh
Bourne-compatible shell script used for building the host build tools
and the
.Nx
system from scratch.
Can be used for both native and cross builds, and should be used instead of
.Xr make 1
for any source tree that is updated and recompiled regularly.
.It Sy crypto/dist/ , dist/ , gnu/dist/
Sources imported verbatim from third parties, without mangling the
existing build structure.
Other source trees in
.Sy bin
through
.Sy usr.sbin
use the
.Nx
.Xr make 1
.Dq reachover
Makefile semantics when building these programs for a native host.
.It Sy distrib/ , etc/
Sources for items used when making a full release snapshot, such as
files installed in
.Sy DESTDIR Ns Pa /etc
on the destination system, boot media, and release notes.
.It Sy tests/ , regress/
Regression test harness.
Can be cross-compiled, but only run natively.
.Pa tests/
uses the
.Xr atf 7
test framework;
.Pa regress/
contains older tests that have not yet been migrated to
.Xr atf 7 .
.It Sy sys/
.Nx
kernel sources.
.It Sy tools/
.Dq Reachover
build structure for the host build tools.
This has a special method of determining out-of-date status.
.It Sy bin/ ... usr.sbin/
Sources to the
.Nx
userland (non-kernel) programs.
If any of these directories are missing, they will be skipped during the build.
.It Sy x11/
.Dq Reachover
build structure for X11R6; the source is in
.Sy X11SRCDIR .
.El
.
.Ss Build tree layout
.
The
.Nx
build tree is described in
.Xr hier 7 ,
and the release layout is described in
.Xr release 7 .
.
.Sh CONFIGURATION
.
.Ss Environment variables
.
Several environment variables control the behaviour of
.Nx
builds.
.
.Bl -tag -width "MAKEOBJDIRPREFIX"
.
.
.It Sy HOST_SH
Path name to a shell available on the host system
and suitable for use during the build.
The
.Nx
build system requires a modern Bourne-like shell
with POSIX-compliant features,
and also requires support for the
.Dq local
keyword to declare local variables in shell functions
(which is a widely-implemented but non-standardised feature).
.Pp
Depending on the host system, a suitable shell may be
.Pa /bin/sh ,
.Pa /usr/xpg4/bin/sh ,
.Pa /bin/ksh
(provided it is a variant of ksh that supports the
.Dq local
keyword,
such as ksh88, but not ksh93),
or
.Pa /usr/local/bin/bash .
.Pp
Most parts of the build require
.Sy HOST_SH
to be an absolute path; however,
.Nm build.sh
allows it to be a simple command name, which will be converted
to an absolute path by searching the
.Sy PATH .
.
.It Sy HOST_CC
Path name to C compiler used to create the toolchain.
.
.It Sy HOST_CXX
Path name to C++ compiler used to create the toolchain.
.
.It Sy MACHINE
Machine type, e.g.,
.Dq macppc .
.
.It Sy MACHINE_ARCH
Machine architecture, e.g.,
.Dq powerpc .
.
.It Sy MAKE
Path name to invoke
.Xr make 1
as.
.
.It Sy MAKEFLAGS
Flags to invoke
.Xr make 1
with.
Note that
.Sy build.sh
ignores the value of
.Sy MAKEFLAGS
passed in the environment, but allows
.Sy MAKEFLAGS
to be set via the
.Fl V
option.
.
.It Sy MAKEOBJDIR
Directory to use as the
.Sy .OBJDIR
for the current directory.
The value is subjected to variable expansion by
.Xr make 1 .
Typical usage is to set this variable to a value involving the use of
.Sq ${.CURDIR:S...}
or
.Sq ${.CURDIR:C...} ,
to derive the value of
.Sy .OBJDIR
from the value of
.Sy .CURDIR .
Used only if
.Sy MAKEOBJDIRPREFIX
is not defined.
.Sy MAKEOBJDIR
can be provided only in the environment or via the
.Fl O
flag of
.Nm build.sh ;
it cannot usefully be set inside a Makefile, including
.Pa mk.conf
or
.Sy ${MAKECONF} .
.
.It Sy MAKEOBJDIRPREFIX
Top level directory of the object directory tree.
The value is subjected to variable expansion by
.Xr make 1 .
.Sy build.sh
will create the
${MAKEOBJDIRPREFIX}
directory if necessary, but if
.Xr make 1
is used without
.Sy build.sh ,
then rules in
.Aq bsd.obj.mk
will abort the build if the
${MAKEOBJDIRPREFIX}
directory does not exist.
If the value is defined and valid, then
${MAKEOBJDIRPREFIX}/${.CURDIR}
is used as the
.Sy .OBJDIR
for the current directory.
The current directory may be read only.
.Sy MAKEOBJDIRPREFIX
can be provided only in the environment or via the
.Fl M
flag of
.Nm build.sh ;
it cannot usefully be set inside a Makefile, including
.Pa mk.conf
or
.Sy ${MAKECONF} .
.
.El
.
.Ss \*qmake\*q variables
.
.de YorN
Can be set to
.Dq yes
or
.Dq no .
..
.de DFLT
.Pp
.Em Default :
..
.de DFLTu
.DFLT
Unset.
..
.de DFLTy
.DFLT
.Dq yes
..
.de DFLTn
.DFLT
.Dq no
..
Several variables control the behavior of
.Nx
builds.
Unless otherwise specified, these variables may be set in
either the process environment or the
.Xr make 1
configuration file specified by
.Sy MAKECONF .
.
.Bl -tag -width "MKCATPAGES"
.
.It Sy BUILDID
Identifier for the build.
The identifier will be appended to
object directory names, and can be consulted in the
.Xr make 1
configuration file in order to set additional build parameters,
such as compiler flags.
.
.It Sy BUILDSEED
GCC uses random numbers when compiling C++ code.
This variable seeds the gcc random number generator using
the -frandom-seed flag with this value.
By default, it is set to NetBSD-(majorversion).
Using a fixed value causes C++ binaries to be the same when
built from the same sources, resulting in identical (reproducible) builds.
Additional information is available in the GCC
documentation of -frandom-seed.
.
.It Sy DESTDIR
Directory to contain the built
.Nx
system.
If set, special options are passed to the compilation tools to
prevent their default use of the host system's
.Sy /usr/include , /usr/lib ,
and so forth.
This pathname must be an absolute path, and should
.Em not
end with a slash
.Pq /
character.
(For installation into the system's root directory, set
.Sy DESTDIR
to an empty string, not to
.Dq / ) .
The directory must reside on a file system which supports long file
names and hard links.
.DFLT
Empty string if
.Sy USETOOLS
is
.Dq yes ;
unset otherwise.
.Pp
.Em Note :
.Sy build.sh
will provide a default of
.Pa destdir. Ns Sy MACHINE
(in the top-level
.Sy .OBJDIR )
unless run in
.Sq expert
mode.
.
.It Sy MAKECONF
The name of the
.Xr make 1
configuration file.
.Em Only settable in the process environment.
.DFLT
.Dq /etc/mk.conf
.
.It Sy MAKEVERBOSE
Level of verbosity of status messages.
Supported values:
.Bl -tag -width xxx
.It 0
No descriptive messages or commands executed by
.Xr make 1
are shown.
.It 1
Brief messages are shown describing what is being done,
but the actual commands executed by
.Xr make 1
are not displayed.
.It 2
Descriptive messages are shown as above (prefixed with a
.Sq # ) ,
and ordinary commands performed by
.Xr make 1
are displayed.
.It 3
In addition to the above, all commands performed by
.Xr make 1
are displayed, even if they would ordinarily have been hidden
through use of the
.Dq \&@
prefix in the relevant makefile.
.It 4
In addition to the above, commands executed by
.Xr make 1
are traced through use of the
.Xr sh 1
.Dq Fl x
flag.
.El
.DFLT
2
.
.It Sy MKCATPAGES
.YorN
Indicates whether preformatted plaintext manual pages will be created
during a build.
.DFLTn
.
.It Sy MKCRYPTO
.YorN
Indicates whether cryptographic code will be included in a build;
provided for the benefit of countries that do not allow strong
cryptography.
Will not affect use of the standard low-security password encryption system,
.Xr crypt 3 .
.DFLTy
.
.It Sy MKDOC
.YorN
Indicates whether system documentation destined for
.Sy DESTDIR Ns Pa /usr/share/doc
will be installed during a build.
.DFLTy
.
.It Sy MKHTML
.YorN
Indicates whether preformatted HTML manual pages will be built
and installed
.DFLTy
.
.It Sy MKHOSTOBJ
.YorN
If set to
.Dq yes ,
then for programs intended to be run on the compile host,
the name, release, and architecture of the host operating system
will be suffixed to the name of the object directory created by
.Dq make obj .
(This allows multiple host systems to compile NetBSD for a single target.)
If set to
.Dq no ,
then programs built to be run on the compile host will use the same
object directory names as programs built to be run on the target.
.DFLTn
.
.It Sy MKINFO
.YorN
Indicates whether GNU Info files, used for the documentation for
most of the compilation tools, will be created and installed during a
build.
.DFLTy
.
.It Sy MKKMOD
.YorN
Indicates whether kernel modules are built and installed.
.DFLTy
.
.It Sy MKLINT
.YorN
Indicates whether
.Xr lint 1
will be run against portions of the
.Nx
source code during the build, and whether lint libraries will be
installed into
.Sy DESTDIR Ns Pa /usr/libdata/lint .
.DFLTy
.
.It Sy MKMAN
.YorN
Indicates whether manual pages will be installed during a build.
.DFLTy
.
.It Sy MKNLS
.YorN
Indicates whether Native Language System locale zone files will be
compiled and installed during a build.
.DFLTy
.
.It Sy MKOBJ
.YorN
Indicates whether object directories will be created when running
.Dq make obj .
If set to
.Dq no ,
then all built files will be located inside the regular source tree.
.DFLTy
.Pp
Note that setting
.Sy MKOBJ
to
.Dq no
is not recommended and may cause problems when updating the tree with
.Xr cvs 1 .
.
.It Sy MKPIC
.YorN
Indicates whether shared objects and libraries will be created and
installed during a build.
If set to
.Dq no ,
the entire built system will be statically linked.
.DFLT
Platform dependent.
As of this writing, all platforms except
.Sy sh3
default to
.Dq yes .
.
.It Sy MKPICINSTALL
.YorN
Indicates whether the
.Xr ar 1
format libraries
.Sy ( lib*_pic.a ) ,
used to generate shared libraries, are installed during a build.
.DFLTy
.
.It Sy MKPROFILE
.YorN
Indicates whether profiled libraries
.Sy ( lib*_p.a )
will be built and installed during a build.
.DFLT
.Dq yes ;
however, some platforms turn off
.Sy MKPROFILE
by default at times due to toolchain problems with profiled code.
.
.It Sy MKREPRO
.YorN
Create reproducable builds.
This enables different switches to make two builds from the same
source tree result in the same build results.
.DFLTn
.
.It Sy MKSHARE
.YorN
Indicates whether files destined to reside in
.Sy DESTDIR Ns Pa /usr/share
will be built and installed during a build.
If set to
.Dq no ,
then all of
.Sy MKCATPAGES , MKDOC , MKINFO , MKMAN ,
and
.Sy MKNLS
will be set to
.Dq no
unconditionally.
.DFLTy
.
.It Sy MKSTRIPIDENT
.YorN
Indicates whether RCS IDs, for use with
.Xr ident 1 ,
should be stripped from program binaries and shared libraries.
.DFLTn
.
.It Sy MKUNPRIVED
.YorN
Indicates whether an unprivileged install will occur.
The user, group, permissions, and file flags, will not be set on
the installed items; instead the information will be appended to
a file called
.Pa METALOG
in
.Sy DESTDIR .
The contents of
.Pa METALOG
are used during the generation of the distribution tar files to ensure
that the appropriate file ownership is stored.
.DFLTn
.
.It Sy MKUPDATE
.YorN
Indicates whether all install operations intended to write to
.Sy DESTDIR
will compare file timestamps before installing, and skip the install
phase if the destination files are up-to-date.
This also has implications on full builds (see next subsection).
.DFLTn
.
.It Sy MKX11
.YorN
Indicates whether X11 is built from
.Sy X11SRCDIR .
.DFLTn
.
.It Sy TOOLDIR
Directory to hold the host tools, once built.
If specified, must be an absolute path.
This directory should be unique to a given host system and
.Nx
source tree.
(However, multiple targets may share the same
.Sy TOOLDIR ;
the target-dependent files have unique names.)  If unset, a default based
on the
.Xr uname 1
information of the host platform will be created in the
.Sy .OBJDIR
of
.Pa src .
.DFLTu
.
.It Sy USETOOLS
Indicates whether the tools specified by
.Sy TOOLDIR
should be used as part of a build in progress.
Must be set to
.Dq yes
if cross-compiling.
.Bl -tag -width "never"
.It Sy yes
Use the tools from
.Sy TOOLDIR .
.It Sy no
Do not use the tools from
.Sy TOOLDIR ,
but refuse to build native compilation tool components that are
version-specific for that tool.
.It Sy never
Do not use the tools from
.Sy TOOLDIR ,
even when building native tool components.
This is similar to the traditional
.Nx
build method, but does
.Em not
verify that the compilation tools in use are up-to-date enough in order
to build the tree successfully.
This may cause build or runtime problems when building the whole
.Nx
source tree.
.El
.DFLT
.Dq yes ,
unless
.Sy TOOLCHAIN_MISSING
is set to
.Dq yes .
.Pp
.Sy USETOOLS
is also set to
.Dq no
when using
.Aq bsd.*.mk
outside the
.Nx
source tree.
.
.It Sy X11SRCDIR
Directory containing the X11R6 source.
If specified, must be an absolute path.
The main X11R6 source is found in
.Sy X11SRCDIR Ns Pa /xfree/xc .
.DFLT
.Sy NETBSDRCDIR Ns Pa /../xsrc ,
if that exists; otherwise
.Pa /usr/xsrc .
.
.It Sy X11FLAVOUR
The style of X11 cross-built, set to either
.Dq Xorg
or
.Dq XFree86 .
.DFLT
.Dq Xorg
on amd64, i386, macppc, shark and sparc64 platforms,
.Dq XFree86
on everything else.
.
.El
.
.Ss \*qmake\*q variables for full builds
These variables only affect the top level
.Dq Makefile
and do not affect manually building subtrees of the
.Nx
source code.
.
.Bl -tag -width "INSTALLWORLDDIR"
.
.It Sy INSTALLWORLDDIR
Location for the
.Dq make installworld
target to install to.
If specified, must be an absolute path.
.DFLT
.Dq /
.
.It Sy MKOBJDIRS
.YorN
Indicates whether object directories will be created automatically
(via a
.Dq make obj
pass) at the start of a build.
.DFLTn
.Pp
If using
.Sy build.sh ,
the default is
.Dq yes .
This may be set back to
.Dq no
by giving
.Sy build.sh
the
.Fl o
option.
.
.It Sy MKUPDATE
.YorN
If set, then in addition to the effects described for
.Sy MKUPDATE=yes
above, this implies the effects of
.Sy NOCLEANDIR
(i.e.,
.Dq make cleandir
is avoided).
.DFLTn
.Pp
If using
.Sy build.sh ,
this may be set by giving the
.Fl u
option.
.
.It Sy NBUILDJOBS
Now obsolete.
Use the
.Xr make 1
option
.Fl j ,
instead.
See below.
.DFLTu
.
.It Sy NOCLEANDIR
If set, avoids the
.Dq make cleandir
phase of a full build.
This has the effect of allowing only changed
files in a source tree to be recompiled.
This can speed up builds when updating only a few files in the tree.
.DFLTu
.Pp
See also
.Sy MKUPDATE .
.
.It Sy NODISTRIBDIRS
If set, avoids the
.Dq make distrib-dirs
phase of a full build.
This skips running
.Xr mtree 8
on
.Sy DESTDIR ,
useful on systems where building as an unprivileged user, or where it is
known that the system-wide mtree files have not changed.
.DFLTu
.
.It Sy NOINCLUDES
If set, avoids the
.Dq make includes
phase of a full build.
This has the effect of preventing
.Xr make 1
from thinking that some programs are out-of-date simply because the
system include files have changed.
However, this option should not be used when updating the entire
.Nx
source tree arbitrarily; it is suggested to use
.Sy MKUPDATE=yes
instead in that case.
.DFLTu
.
.It Sy RELEASEDIR
If set, specifies the directory to which a
.Xr release 7
layout will be written at the end of a
.Dq make release .
If specified, must be an absolute path.
.DFLTu
.Pp
.Em Note :
.Sy build.sh
will provide a default of
.Pa releasedir
(in the top-level
.Sy .OBJDIR )
unless run in
.Sq expert
mode.
.
.El
.
.Sh BUILDING
.
.Ss \*qmake\*q command line options
This is not a summary of all the options available to
.Xr make 1 ;
only the options used most frequently with
.Nx
builds are listed here.
.
.Bl -tag -width "var=value"
.
.It Fl j Ar njob
Run up to
.Ar njob
.Xr make 1
subjobs in parallel.
Makefiles should use .WAIT or have explicit dependencies
as necessary to enforce build ordering.
.
.It Fl m Ar dir
Specify the default directory for searching for system Makefile
segments, mainly the
.Aq bsd.*.mk
files.
When building any full
.Nx
source tree, this should be set to the
.Dq share/mk
directory in the source tree.
This is set automatically when building from the top level, or when using
.Sy build.sh .
.
.It Fl n
Display the commands that would have been executed, but do not
actually execute them.
This will still cause recursion to take place.
.
.It Fl V Ar var
Print
.Xr make 1 Ns 's
idea of the value of
.Ar var .
Does not build any targets.
.
.It Em var=value
Set the variable
.Em var
to
.Em value ,
overriding any setting specified by the process environment, the
.Sy MAKECONF
configuration file, or the system Makefile segments.
.
.El
.
.Ss \*qmake\*q targets
.
These default targets may be built by running
.Xr make 1
in any subtree of the
.Nx
source code.
It is recommended that none of these be used from the top
level Makefile; as a specific exception,
.Dq make obj
and
.Dq make cleandir
are useful in that context.
.
.Bl -tag -width "dependall"
.
.It Sy all
Build programs, libraries, and preformatted documentation.
.
.It Sy clean
Remove program and library object code files.
.
.It Sy cleandir
Same as
.Sy clean ,
but also remove preformatted documentation, dependency files generated
by
.Dq make depend ,
and any other files known to be created at build time.
.
.It Sy depend
Create dependency files
.Sy ( .depend )
containing more detailed information about the dependencies of source
code on header files.
Allows programs to be recompiled automatically when a dependency changes.
.
.It Sy dependall
Does a
.Dq make depend
immediately followed by a
.Dq make all .
This improves cache locality of the build since both passes read the source
files in their entirety.
.
.It Sy distclean
Synonym for
.Sy cleandir .
.
.It Sy includes
Build and install system header files.
Typically needed before any system libraries or programs can be built.
.
.It Sy install
Install programs, libraries, and documentation into
.Sy DESTDIR .
Few files will be installed to
.Sy DESTDIR Ns Pa /dev ,
.Sy DESTDIR Ns Pa /etc ,
.Sy DESTDIR Ns Pa /root
or
.Sy DESTDIR Ns Pa /var
in order to prevent user supplied configuration data from being overwritten.
.It Sy lint
Run
.Xr lint 1
against the C source code, where appropriate, and generate
system-installed lint libraries.
.
.It Sy obj
Create object directories to be used for built files, instead of
building directly in the source tree.
.
.It Sy tags
Create
.Xr ctags 1
searchable function lists usable by the
.Xr ex 1
and
.Xr vi 1
text editors.
.
.El
.
.Ss \*qmake\*q targets for the top level
.
Additional
.Xr make 1
targets are usable specifically from the top source level to facilitate
building the entire
.Nx
source tree.
.
.Bl -tag -width "distribution"
.
.It Sy build
Build the entire
.Nx
system (except the kernel).
This orders portions of the source tree such that prerequisites
will be built in the proper order.
.
.It Sy distribution
Do a
.Dq make build ,
and then install a full distribution (which does not include a kernel) into
.Sy DESTDIR ,
including files in
.Sy DESTDIR Ns Pa /dev ,
.Sy DESTDIR Ns Pa /etc ,
.Sy DESTDIR Ns Pa /root
and
.Sy DESTDIR Ns Pa /var .
.
.It Sy buildworld
As per
.Dq make distribution ,
except that it ensures that
.Sy DESTDIR
is not the root directory.
.
.It Sy installworld
Install the distribution from
.Sy DESTDIR
to
.Sy INSTALLWORLDDIR ,
which defaults to the root directory.
Ensures that
.Sy INSTALLWORLDDIR
is not the root directory if cross compiling.
.Pp
The
.Sy INSTALLSETS
environment variable may be set to a space-separated list of
distribution sets to be installed.
By default, all sets except
.Dq etc
and
.Dq xetc
are installed, so most files in
.Sy INSTALLWORLDDIR Ns Pa /etc
will not be installed or modified.
.Pp
.Em Note :
Before performing this operation with
.Sy INSTALLWORLDDIR Ns = Ns Pa / ,
it is highly recommended that you upgrade your kernel and reboot.
After performing this operation,
it is recommended that you use
.Xr etcupdate 8
to update files in
.Sy INSTALLWORLDDIR Ns Pa /etc ,
and
.Xr postinstall 8
to check for or fix inconsistencies.
.It Sy sets
Create distribution sets from
.Sy DESTDIR
into
.Sy RELEASEDIR/RELEASEMACHINEDIR Ns Pa /binary/sets .
Should be run after
.Dq make distribution ,
as
.Dq make build
alone does not install all of the required files.
.
.It Sy sourcesets
Create source sets of the source tree into
.Sy RELEASEDIR Ns Pa /source/sets .
.
.It Sy syspkgs
Create syspkgs from
.Sy DESTDIR
into
.Sy RELEASEDIR/RELEASEMACHINEDIR Ns Pa /binary/syspkgs .
Should be run after
.Dq make distribution ,
as
.Dq make build
alone does not install all of the required files.
.
.It Sy release
Do a
.Dq make distribution ,
build kernels, distribution media, and install sets
(this as per
.Dq make sets ) ,
and
then package the system into a standard release layout as described by
.Xr release 7 .
This requires that
.Sy RELEASEDIR
be set (see above).
.
.It Sy iso-image
Create a
.Nx
installation CD-ROM image in the
.Sy RELEASEDIR Ns Pa /iso
directory.
The CD-ROM file system will have a layout as described in
.Xr release 7 .
.Pp
For most machine types, the CD-ROM will be bootable, and will automatically
run the
.Xr sysinst 8
menu-based installation program, which can be used to install or upgrade a
.Nx
system.
Bootable CD-ROMs also contain tools that may be useful in
repairing a damaged
.Nx
installation.
.Pp
Before
.Dq make iso-image
is attempted, RELEASEDIR must be populated by
.Dq make release
or equivalent.
.Pp
Note that other, smaller, CD-ROM images may be created in the
.Sy RELEASEDIR/RELEASEMACHINEDIR Ns Pa /installation/cdrom
directory by
.Dq "make release" .
These smaller images usually contain the same tools as the larger images in
.Sy RELEASEDIR Ns Pa /iso ,
but do not contain additional content such as the distribution sets.
.Pp
Note that the mac68k port still uses an older method of creating
CD-ROM images.
This requires the
.Xr mkisofs 1
utility, which is not part of
.Nx ,
but which can be installed from
.Pa pkgsrc/sysutils/cdrtools .
.
.It Sy iso-image-source
Create a
.Nx
installation CD-ROM image in the
.Sy RELEASEDIR Ns Pa /iso
directory.
The CD-ROM file system will have a layout as described in
.Xr release 7 .
It will have top level directories for the machine type and source.
.Pp
For most machine types, the CD-ROM will be bootable, and will automatically
run the
.Xr sysinst 8
menu-based installation program, which can be used to install or upgrade a
.Nx
system.
Bootable CD-ROMs also contain tools that may be useful in
repairing a damaged
.Nx
installation.
.Pp
Before
.Dq make iso-image-source
is attempted, RELEASEDIR must be populated by
.Dq make sourcesets release
or equivalent.
.Pp
Note that other, smaller, CD-ROM images may be created in the
.Sy RELEASEDIR/RELEASEMACHINEDIR Ns Pa /installation/cdrom
directory by
.Dq make release .
These smaller images usually contain the same tools as the larger images in
.Sy RELEASEDIR Ns Pa /iso ,
but do not contain additional content such as the distribution sets.
.Pp
Note that the mac68k port still uses an older method of creating
CD-ROM images.
This requires the
.Xr mkisofs 1
utility, which is not part of
.Nx ,
but which can be installed from
.Pa pkgsrc/sysutils/cdrtools .
.
.It Sy install-image
Create a bootable
.Nx
installation disk image in the
.Sy RELEASEDIR/RELEASEMACHINEDIR Ns Pa /installation/installimage
directory.
The installation disk image is suitable for copying to
bootable USB flash memory sticks, etc., for machines which
are able to boot from such devices.
The file system in the bootable disk image will have a layout
as described in
.Xr release 7 .
.Pp
The installation image is bootable, and will automatically
run the
.Xr sysinst 8
menu-based installation program, which can be used to install or upgrade a
.Nx
system.
The image also contains tools that may be
useful in repairing a damaged
.Nx
installation.
.Pp
Before
.Dq make install-image
is attempted,
.Sy RELEASEDIR
must be populated by
.Dq make release
or equivalent.
The build must have been performed with
.Sy MKUNPRIVED=yes
because
.Dq make install-image
relies on information in
.Sy DESTDIR Ns Pa /METALOG .
.
.It live-image
Create
.Nx
live images in the
.Sy RELEASEDIR/RELEASEMACHINEDIR Ns Pa /installation/liveimage
directory.
The live image contains all necessary files
to boot
.Nx
up to multi-user mode, including all files
which should be extracted during installation,
.Nx
disklabel, bootloaders, etc.
.Pp
The live image is suitable for use as a disk image in
virtual machine environments such as QEMU,
and also useful to boot
.Nx
from a USB flash memory stick
on a real machine, without the need for installation.
.Pp
Before
.Dq make live-image
is attempted, RELEASEDIR must be populated by
.Dq make release
or equivalent.
The build must have been performed with
.Sy MKUNPRIVED=yes
because
.Dq make install-image
relies on information in
.Sy DESTDIR Ns Pa /METALOG .
.
.It Sy regression-tests
Can only be run after building the regression tests in the directory
.Dq regress .
Runs those compiled regression tests on the local host.
Note that most tests are now managed instead using
.Xr atf 7 ;
this target should probably run those as well but currently does not.
.
.El
.
.Ss The \*qbuild.sh\*q script
.
This script file is a shell script designed to build the
entire
.Nx
system on any host with a suitable modern shell and some common
utilities.
The required shell features are described under the
.Sy HOST_SH
variable.
.Pp
If a host system's default shell does support the required
features, then we suggest that you explicitly specify
a suitable shell using a command like
.Bd -unfilled -offset indent
.Li /path/to/suitable/shell build.sh Op Ar options
.Ed
.Pp
The above command will usually enable
.Nm build.sh
to automatically set
.Sy HOST_SH Ns Cm = Ns Pa /path/to/suitable/shell ,
but if that fails, then the following set of commands may be used instead:
.Bd -unfilled -offset indent
.Li HOST_SH= Ns Va /path/to/suitable/shell
.Li export HOST_SH
.Li ${HOST_SH} build.sh Op Ar options
.Ed
.Pp
If
.Sy build.sh
detects that it is being executed under an unsuitable shell, it attempts
to exec a suitable shell instead, or prints an error message.
If
.Sy HOST_SH
is not set explicitly, then
.Nm build.sh
sets a default using heuristics dependent on the host platform,
or from the shell under which
.Nm build.sh
is executed (if that can be determined),
or using the first copy of
.Pa sh
found in
.Sy PATH .
.Pp
All cross-compile builds, and most native builds, of the entire system
should make use of
.Sy build.sh
rather than just running
.Dq make .
This way, the
.Xr make 1
program will be bootstrapped properly, in case the host system has an
older or incompatible
.Dq make
program.
.Pp
When compiling the entire system via
.Sy build.sh ,
many
.Xr make 1
variables are set for you in order to help encapsulate the build
process.
In the list of options below, variables that are automatically set by
.Sy build.sh
are noted where applicable.
.
.Pp
The following operations are supported by
.Sy build.sh :
.
.Bl -tag -width "distribution"
.
.It Sy build
Build the system as per
.Dq make build .
Before the main part of the build commences, this command runs the
.Sy obj
operation (unless the
.Fl o
option is given),
.Dq make cleandir
(unless the
.Fl u
option is given),
and the
.Sy tools
operation.
.
.It Sy distribution
Build a full distribution as per
.Dq make distribution .
This command first runs the
.Sy build
operation.
.
.It Sy release
Build a full release as per
.Dq make release .
This command first runs the
.Sy distribution
operation.
.
.It Sy makewrapper
Create the
.Sy \*[toolprefix]make-MACHINE
wrapper.
This operation is automatically performed for any of the other
operations.
.
.It Sy cleandir
Perform
.Dq make cleandir .
.
.It Sy obj
Perform
.Dq make obj .
.
.It Sy tools
Build and install the host tools from
.Pa src/tools .
This command will first run
.Dq make obj
and
.Dq make cleandir
in the
.Pa tools
subdirectory unless the
.Fl o
or
.Fl u
options (respectively) are given.
.
.It Sy install Ns = Ns Ar idir
Install the contents of
.Sy DESTDIR
to
.Ar idir ,
using
.Dq make installworld .
Note that files that are part of the
.Dq etc
or
.Dq xetc
sets will not be installed, unless overridden by
the INSTALLSETS environment variable.
.
.It Sy kernel Ns = Ns Ar kconf
Build a new kernel.
The
.Ar kconf
argument is the name of a configuration file suitable
for use by
.Xr config 1 .
If
.Ar kconf
does not contain any
.Sq /
characters, the configuration file is expected to be found in the
.Sy KERNCONFDIR
directory, which is typically
.Sy sys/arch/MACHINE/conf .
The new kernel will be built in a subdirectory of
.Sy KERNOBJDIR ,
which is typically
.Sy sys/arch/MACHINE/compile
or an associated object directory.
.Pp
This command does
.Em not
imply the
.Sy tools
command; run the
.Sy tools
command first unless it is
.Em certain
that the tools already exist and are up to date.
.Pp
This command will run
.Dq make cleandir
on the kernel in question first unless the
.Fl u
option is given.
.
.It Sy modules
This command will build kernel modules and install them into
.Sy DESTDIR .
.
.It Sy releasekernel Ns = Ns Ar kconf
Install a
.Xr gzip 1 Ns ed
copy of the kernel previously built by
.Sy kernel Ns = Ns Ar kconf
into
.Sy RELEASEDIR/RELEASEMACHINEDIR Ns Pa /binary/kernel ,
usually as
.Pa netbsd- Ns Ar kconf Ns Pa .gz ,
although the
.Dq Pa netbsd
prefix is determined from the
.Dq Sy config
directives in
.Ar kconf .
.
.It Sy sets
Perform
.Dq make sets .
.
.It Sy sourcesets
Perform
.Dq make sourcesets .
.
.It Sy syspkgs
Perform
.Dq make syspkgs .
.
.It Sy iso-image
Perform
.Dq make iso-image .
.
.It Sy iso-image-source
Perform
.Dq make iso-image-source .
.
.It Sy install-image
Perform
.Dq make install-image .
.
.It Sy live-image
Perform
.Dq make live-image .
.
.El
.
.Pp
The following command line options alter the behaviour of the
.Sy build.sh
operations described above:
.
.Bl -tag -width "-T tools"
.
.It Fl a Ar arch
Set the value of
.Sy MACHINE_ARCH
to
.Ar arch .
.
.It Fl B Ar buildid
Set the value of
.Sy BUILDID
to
.Ar buildid .
This will also append the build identifier to the name of the
.Dq make
wrapper script so that the resulting name is of the form
.Dq Sy \*[toolprefix]make-MACHINE-BUILDID .
.
.It Fl C Ar cdextras
Append
.Ar cdextras
to the
.Sy CDEXTRA
variable,
which is a space-separated list of files or directories that will be
added to the CD-ROM image that may be create by the
.Dq iso-image
or
.Dq iso-image-source
operations.
Files will be added to the root of the CD-ROM image, whereas directories
will be copied recursively.
If relative paths are specified, they will be converted to
absolute paths before being used.
Multiple paths may be specified via multiple
.Fl C
options, or via a single option whose argument contains multiple
space-separated paths.
.
.It Fl D Ar dest
Set the value of
.Sy DESTDIR
to
.Ar dest .
If a relative path is specified, it will be converted to an
absolute path before being used.
.
.It Fl E
Set
.Sq expert
mode.
This overrides various sanity checks, and allows:
.Sy DESTDIR
does not have to be set to a non-root path for builds,
and
.Sy MKUNPRIVED=yes
does not have to be set when building as a non-root user.
.Pp
.Em Note :
It is highly recommended that you know what you are doing when
you use this option.
.
.It Fl h
Print a help message.
.
.It Fl j Ar njob
Run up to
.Ar njob
.Xr make 1
subjobs in parallel;
passed through to
.Xr make 1 .
If you see failures for reasons other than running out of memory
while using
.Sy build.sh
with
.Fl j ,
please save complete build logs
so the failures can be analyzed.
.Pp
To achieve the fastest builds,
.Fl j
values between (1 + the number of CPUs) and (2 * the number of CPUs)
are recommended.
Use lower values on machines with limited memory or I/O bandwidth.
.
.It Fl M Ar obj
Set
.Sy MAKEOBJDIRPREFIX
to
.Ar obj .
Unsets
.Sy MAKEOBJDIR .
See
.Dq Fl O obj
for more information.
.Pp
For instance, if the source directory is
.Pa /usr/src ,
a setting of
.Dq Fl M Pa /usr/obj
will place build-time files under
.Pa /usr/obj/usr/src/bin ,
.Pa /usr/obj/usr/src/lib ,
.Pa /usr/obj/usr/src/usr.bin ,
and so forth.
.Pp
If a relative path is specified, it will be converted to an
absolute path before being used.
.Sy build.sh
imposes the restriction that the argument to the
.Fl M
option must not begin with a
.Dq \&$
(dollar sign)
character; otherwise it would be too difficult
to determine whether the value is an absolute or a relative path.
If the directory does not already exist,
.Sy build.sh
will create it.
.
.It Fl m Ar mach
Set the value of
.Sy MACHINE
to
.Ar mach ,
except in some special cases listed below.
This will also override any value of
.Sy MACHINE_ARCH
in the process environment with a value deduced from
.Ar mach ,
unless
.Fl a
is specified.
All cross builds require
.Fl m ,
but if unset on a NetBSD host, the host's value of MACHINE will be
detected and used automatically.
.Pp
Some machines support multiple values for
.Sy MACHINE_ARCH .
The following special cases for the
.Ar mach
argument are defined to set the listed values of
.Sy MACHINE
and
.Sy MACHINE_ARCH :
.Bl -column "evbmips-el" "MACHINE" "MACHINE_ARCH" -offset indent
.It Sy mach Ta Sy "MACHINE" Ta Sy "MACHINE_ARCH"
.It evbarm Ta evbarm Ta (not set)
.It evbarm-eb Ta evbarm Ta armeb
.It evbarm-el Ta evbarm Ta arm
.It evbmips Ta evbmips Ta (not set)
.It evbmips-eb Ta evbmips Ta mipseb
.It evbmips-el Ta evbmips Ta mipsel
.It evbsh3 Ta evbsh3 Ta (not set)
.It evbsh3-eb Ta evbsh3 Ta sh3eb
.It evbsh3-el Ta evbsh3 Ta sh3el
.It sbmips Ta sbmips Ta (not set)
.It sbmips-eb Ta sbmips Ta mipseb
.It sbmips-el Ta sbmips Ta mipsel
.El
.
.It Fl N Ar noiselevel
Set the
.Dq noisyness
level of the build, by setting
.Sy MAKEVERBOSE
to
.Ar noiselevel .
.
.It Fl n
Show the commands that would be executed by
.Sy build.sh ,
but do not make any changes.
This is similar in concept to
.Dq make -n .
.
.It Fl O Ar obj
Create an appropriate transform macro for
.Sy MAKEOBJDIR
that will place the built object files under
.Ar obj .
Unsets
.Sy MAKEOBJDIRPREFIX .
.Pp
For instance, a setting of
.Dq Fl O Pa /usr/obj
will place build-time files under
.Pa /usr/obj/bin ,
.Pa /usr/obj/lib ,
.Pa /usr/obj/usr.bin ,
and so forth.
.Pp
If a relative path is specified, it will be converted to an
absolute path before being used.
.Sy build.sh
imposes the restriction that the argument to the
.Fl O
option must not contain a
.Dq \&$
(dollar sign)
character.
If the directory does not already exist,
.Sy build.sh
will create it.
.Pp
In normal use, exactly one of the
.Fl M
or
.Fl O
options should be specified.
If neither
.Fl M
nor
.Fl O
is specified, then a default object directory will be chosen
according to rules in
.Aq bsd.obj.mk .
Relying on this default is not recommended because
it is determined by complex rules that are influenced
by the values of several variables and
by the location of the source directory.
.It Fl o
Set the value of
.Sy MKOBJDIRS
to
.Dq no .
Otherwise, it will be automatically set to
.Dq yes .
This default is opposite to the behaviour when not using
.Sy build.sh .
.
.It Fl R Ar rel
Set the value of
.Sy RELEASEDIR
to
.Ar rel .
If a relative path is specified, it will be converted to an
absolute path before being used.
.
.It Fl r
Remove the contents of
.Sy DESTDIR
and
.Sy TOOLDIR
before building (provides a clean starting point).
This will skip deleting
.Sy DESTDIR
if building on a native system to the root directory.
.
.It Fl S Ar seed
Change the value of
.Sy BUILDSEED
to
.Ar seed .
This should rarely be necessary.
.
.It Fl T Ar tools
Set the value of
.Sy TOOLDIR
to
.Ar tools .
If a relative path is specified, it will be converted to an
absolute path before being used.
If set, the bootstrap
.Dq make
will only be rebuilt if the source files for
.Xr make 1
have changed.
.
.It Fl U
Set
.Sy MKUNPRIVED=yes .
.
.It Fl u
Set
.Sy MKUPDATE=yes .
.
.It Xo
.Fl V
.Sm off
.Ar var
.Li =
.Op Ar value
.Sm on
.Xc
Set the environment variable
.Ar var
to an optional
.Ar value .
This is propagated to the
.Sy \*[toolprefix]make
wrapper.
.
.It Fl w Ar wrapper
Create the
.Sy \*[toolprefix]make
wrapper script (see below) in a custom location,
specified by
.Ar wrapper .
This allows, for instance, to place the wrapper in
.Sy PATH
automatically.
Note that
.Ar wrapper
is the full name of the file, not just a directory name.
If a relative path is specified, it will be converted to an
absolute path before being used.
.
.It Fl X Ar x11src
Set the value of
.Sy X11SRCDIR
to
.Ar x11src .
If a relative path is specified, it will be converted to an
absolute path before being used.
.
.It Fl x
Set
.Sy MKX11=yes .
.
.It Fl Z Ar var
Unset ("zap") the environment variable
.Ar var .
This is propagated to the
.Sy \*[toolprefix]make
wrapper.
.
.El
.
.Ss The \*q\*[toolprefix]make-MACHINE\*q wrapper script
.
If using the
.Sy build.sh
script to build
.Nx ,
a
.Sy \*[toolprefix]make-MACHINE
script will be created in
.Sy TOOLDIR/bin
upon the first build to assist in building subtrees on a cross-compile
host.
.Pp
.Sy \*[toolprefix]make-MACHINE
can be invoked in lieu of
.Xr make 1 ,
and will instead call the up-to-date version of
.Dq \*[toolprefix]make
installed into
.Sy TOOLDIR/bin
with several key variables pre-set, including
.Sy MACHINE , MACHINE_ARCH ,
and
.Sy TOOLDIR .
.Sy \*[toolprefix]make-MACHINE
will also set variables specified with
.Fl V ,
and unset variables specified with
.Fl Z .
.Pp
This script can be symlinked into a directory listed in
.Sy PATH ,
or called with an absolute path.
.
.Sh EXAMPLES
.
.Bl -enum
.
.It
.Li "% ./build.sh [options] tools kernel=GENERIC"
.Pp
Build a new toolchain, and use the new toolchain to
configure and build a new GENERIC kernel.
.
.It
.Li "% ./build.sh [options] -U distribution"
.Pp
Using unprivileged mode,
build a complete distribution to a
.Sy DESTDIR
directory that
.Sy build.sh
selects (and will display).
.
.It
.Li "# ./build.sh [options] -U install=/"
.Pp
As root, install to
.Pa /
the distribution that was built
by example 2.
Even though this is run as root,
.Fl U
is required so that the permissions stored in
.Sy DESTDIR Ns Pa /METALOG
are correctly applied to the files as they're copied to
.Pa / .
.
.It
.Li "% ./build.sh [options] -U -u release"
.Pp
Using unprivileged mode,
build a complete release to
.Sy DESTDIR
and
.Sy RELEASEDIR
directories that
.Sy build.sh
selects (and will display).
.Sy MKUPDATE=yes
.Pq Fl u
is set to prevent the
.Dq make cleandir ,
so that if this is run after example 2, it doesn't need to redo that
portion of the release build.
.El
.
.Sh OBSOLETE VARIABLES
.
.Bl -tag -width "NBUILDJOBS"
.
.It Sy NBUILDJOBS
Use the
.Xr make 1
option
.Fl j
instead.
.
.It Sy USE_NEW_TOOLCHAIN
The new toolchain is now the default.
To disable, use
.Sy TOOLCHAIN_MISSING=yes .
.El
.Sh SEE ALSO
.Xr make 1 ,
.Xr hier 7 ,
.Xr release 7 ,
.Xr etcupdate 8 ,
.Xr postinstall 8 ,
.Xr sysinst 8 ,
.Pa pkgsrc/sysutils/cdrtools
.
.Sh HISTORY
.
The
.Nm build.sh
based build scheme was introduced for
.Nx 1.6
as
.Sy USE_NEW_TOOLCHAIN ,
and re-worked to
.Sy TOOLCHAIN_MISSING
after that.