diff --git a/BUILDING b/BUILDING new file mode 100644 index 000000000000..f46c41f8eb7d --- /dev/null +++ b/BUILDING @@ -0,0 +1,1115 @@ +BUILDING(8) System Manager's Manual BUILDING(8) + +NAME + BUILDING - Procedure for building NetBSD from source code. + +REQUIREMENTS + NetBSD is designed to be buildable on most POSIX-compliant host systems. + The basic build procedure is the same whether compiling natively (on the + same NetBSD architecture) or cross compiling (on another architecture or + OS). + + This source tree contains a special subtree, "tools", which uses the host + system to create a build toolchain for the target architecture. The host + system must have at least C and C++ compilers in order to create the + toolchain (make is not required); all other tools are created as part of + the NetBSD build process. (See the environment variables section below + if you need to override or manually select your compilers.) + +FILES + Source tree layout + doc/BUILDING.mdoc + This document (in -mdoc troff format; the original copy). + + BUILDING This document (in plaintext). + + tools/compat/README + Special notes for cross-hosting a NetBSD build on non- + NetBSD platforms. + + Makefile The main Makefile for NetBSD; should only be run for + native builds with an appropriately up-to-date version of + NetBSD make(1). Intended for expert use with knowlege of + its shortcomings, it has been superseded by the build.sh + shell script as the recommended means for building NetBSD. + + UPDATING Special notes for updating from an earlier revision of + NetBSD. It is important to read this file before every + build of an updated source tree. + + build.sh Bourne-compatible shell script used for building the host + build tools and the NetBSD system from scratch. Can be + used for both native and cross builds, and should be used + instead of make(1) as it performs additional checks to + prevent common issues going undetected, such as building + with an outdated version of make(1). + + crypto/dist/, dist/, gnu/dist/ + Sources imported verbatim from third parties, without + mangling the existing build structure. Other source trees + in bin through usr.sbin use the NetBSD make(1) "reachover" + Makefile semantics when building these programs for a + native host. + + external, sys/external + Sources and build infrastructure for components imported + (mostly) unchanged from upstream maintainers, sorted by + applicable license. This is (slowly) replacing the + crypto/dist, dist, and gnu/dist directories. + + distrib/, etc/ + Sources for items used when making a full release + snapshot, such as files installed in DESTDIR/etc on the + destination system, boot media, and release notes. + + tests/, regress/ + Regression test harness. Can be cross-compiled, but only + run natively. tests/ uses the atf(7) test framework; + regress/ contains older tests that have not yet been + migrated to atf(7). + + sys/ NetBSD kernel sources. + + tools/ "Reachover" build structure for the host build tools. + This has a special method of determining out-of-date + status. + + bin/ ... usr.sbin/ + Sources to the NetBSD userland (non-kernel) programs. If + any of these directories are missing, they will be skipped + during the build. + + external/mit/xorg/ + "Reachover" build structure for modular Xorg; the source + is in X11SRCDIR. + + extsrc/ "Reachover" build structure for externally added programs + and libraries; the source is in EXTSRCSRCDIR. + + Build tree layout + The NetBSD build tree is described in hier(7), and the release layout is + described in release(7). + +CONFIGURATION + Environment variables + Several environment variables control the behaviour of NetBSD builds. + + HOST_SH Path name to a shell available on the host system and + suitable for use during the build. The NetBSD build + system requires a modern Bourne-like shell with POSIX- + compliant features, and also requires support for the + "local" keyword to declare local variables in shell + functions (which is a widely-implemented but non- + standardised feature). + + Depending on the host system, a suitable shell may be + /bin/sh, /usr/xpg4/bin/sh, /bin/ksh (provided it is a + variant of ksh that supports the "local" keyword, such + as ksh88, but not ksh93), or /usr/local/bin/bash. + + Most parts of the build require HOST_SH to be an + absolute path; however, build.sh allows it to be a + simple command name, which will be converted to an + absolute path by searching the PATH. + + HOST_CC Path name to C compiler used to create the toolchain. + + HOST_CXX Path name to C++ compiler used to create the toolchain. + + MACHINE Machine type, e.g., "macppc". + + MACHINE_ARCH Machine architecture, e.g., "powerpc". + + MAKE Path name to invoke make(1) as. + + MAKEFLAGS Flags to invoke make(1) with. Note that build.sh + ignores the value of MAKEFLAGS passed in the + environment, but allows MAKEFLAGS to be set via the -V + option. + + MAKEOBJDIR Directory to use as the .OBJDIR for the current + directory. The value is subjected to variable + expansion by make(1). Typical usage is to set this + variable to a value involving the use of + `${.CURDIR:S...}' or `${.CURDIR:C...}', to derive the + value of .OBJDIR from the value of .CURDIR. Used only + if MAKEOBJDIRPREFIX is not defined. MAKEOBJDIR can be + provided only in the environment or via the -O flag of + build.sh; it cannot usefully be set inside a Makefile, + including mk.conf or ${MAKECONF}. + + MAKEOBJDIRPREFIX Top level directory of the object directory tree. The + value is subjected to variable expansion by make(1). + build.sh will create the ${MAKEOBJDIRPREFIX} directory + if necessary, but if make(1) is used without build.sh, + then rules in will abort the build if the + ${MAKEOBJDIRPREFIX} directory does not exist. If the + value is defined and valid, then + ${MAKEOBJDIRPREFIX}/${.CURDIR} is used as the .OBJDIR + for the current directory. The current directory may + be read only. MAKEOBJDIRPREFIX can be provided only in + the environment or via the -M flag of build.sh; it + cannot usefully be set inside a Makefile, including + mk.conf or ${MAKECONF}. + + "make" variables + Several variables control the behavior of NetBSD builds. Unless + otherwise specified, these variables may be set in either the process + environment or the make(1) configuration file specified by MAKECONF. + + BUILDID Identifier for the build. If set, this should be a short + string that is suitable for use as part of a file or + directory name. The identifier will be appended to object + directory names, and can be consulted in the make(1) + configuration file in order to set additional build + parameters, such as compiler flags. It will also be used as + part of the kernel version string, which can be printed by + "uname -v". + + Default: Unset. + + BUILDINFO This may be a multi-line string containing information about + the build. This will appear in DESTDIR/etc/release, and it + will be stored in the buildinfo variable in any kernels that + are built. When such kernels are booted, the sysctl(7) + kern.buildinfo variable will report this value. The string + may contain backslash escape sequences, such as "\\" + (representing a backslash character) and "\n" (representing a + newline). + + Default: Unset. + + BUILDSEED GCC uses random numbers when compiling C++ code. This + variable seeds the gcc random number generator using the + -frandom-seed flag with this value. By default, it is set to + NetBSD-(majorversion). Using a fixed value causes C++ + binaries to be the same when built from the same sources, + resulting in identical (reproducible) builds. Additional + information is available in the GCC documentation of + -frandom-seed. + + DESTDIR Directory to contain the built NetBSD system. If set, + special options are passed to the compilation tools to + prevent their default use of the host system's /usr/include, + /usr/lib, and so forth. This pathname must be an absolute + path, and should not end with a slash (/) character. (For + installation into the system's root directory, set DESTDIR to + an empty string, not to "/"). The directory must reside on a + file system which supports long file names and hard links. + + Default: Empty string if USETOOLS is "yes"; unset otherwise. + + Note: build.sh will provide a default of destdir.MACHINE (in + the top-level .OBJDIR) unless run in `expert' mode. + + EXTSRCSRCDIR + Directory containing sources of externally added programs and + libraries. If specified, must be an absolute path. + + Default: NETBSDRCDIR/../extsrc, if that exists; otherwise + /usr/extsrc. + + MAKECONF The name of the make(1) configuration file. Only settable in + the process environment. + + Default: "/etc/mk.conf" + + MAKEVERBOSE + Level of verbosity of status messages. Supported values: + + 0 No descriptive messages or commands executed by make(1) + are shown. + + 1 Brief messages are shown describing what is being done, + but the actual commands executed by make(1) are not + displayed. + + 2 Descriptive messages are shown as above (prefixed with a + `#'), and ordinary commands performed by make(1) are + displayed. + + 3 In addition to the above, all commands performed by + make(1) are displayed, even if they would ordinarily + have been hidden through use of the "@" prefix in the + relevant makefile. + + 4 In addition to the above, commands executed by make(1) + are traced through use of the sh(1) "-x" flag. + + Default: 2 + + MKCATPAGES Can be set to "yes" or "no". Indicates whether preformatted + plaintext manual pages will be created during a build. + + Default: "no" + + MKCROSSGDB Can be set to "yes" or "no". Create a cross-gdb as a host + tool. + + Default: "no" + + MKDEBUG Can be set to "yes" or "no". Indicates whether debug + information should be generated for all userland binaries + compiled. The result is collected as an additional debug.tgz + and xdebug.tgz set and installed in /usr/libdata/debug. + + Default: "no" + + MKDEBUGLIB Can be set to "yes" or "no". Indicates whether debug + information (see MKDEBUG) should also be generated for all + libraries build. + + Default: "no" + + MKDOC Can be set to "yes" or "no". Indicates whether system + documentation destined for DESTDIR/usr/share/doc will be + installed during a build. + + Default: "yes" + + MKEXTSRC Can be set to "yes" or "no". Indicates whether extsrc is + built from EXTSRCSRCDIR. + + Default: "no" + + MKHTML Can be set to "yes" or "no". Indicates whether preformatted + HTML manual pages will be built and installed + + Default: "yes" + + MKHOSTOBJ Can be set to "yes" or "no". If set to "yes", then for + programs intended to be run on the compile host, the name, + release, and architecture of the host operating system will + be suffixed to the name of the object directory created by + "make obj". (This allows multiple host systems to compile + NetBSD for a single target.) If set to "no", then programs + built to be run on the compile host will use the same object + directory names as programs built to be run on the target. + + Default: "no" + + MKINFO Can be set to "yes" or "no". Indicates whether GNU Info + files, used for the documentation for most of the compilation + tools, will be created and installed during a build. + + Default: "yes" + + MKKDEBUG Can be set to "yes" or "no". Force generation of full-debug + symbol versions of all kernels compiled. Alongside of the + netbsd kernel file, an unstripped version netbsd.gdb is + created. This is useful if a cross-gdb is built as well (see + MKCROSSGDB). + + Default: "no" + + MKKMOD Can be set to "yes" or "no". Indicates whether kernel + modules are built and installed. + + Default: "yes" + + MKLINT Can be set to "yes" or "no". Indicates whether lint(1) will + be run against portions of the NetBSD source code during the + build, and whether lint libraries will be installed into + DESTDIR/usr/libdata/lint. + + Default: "yes" + + MKMAN Can be set to "yes" or "no". Indicates whether manual pages + will be installed during a build. + + Default: "yes" + + MKNLS Can be set to "yes" or "no". Indicates whether Native + Language System locale zone files will be compiled and + installed during a build. + + Default: "yes" + + MKOBJ Can be set to "yes" or "no". Indicates whether object + directories will be created when running "make obj". If set + to "no", then all built files will be located inside the + regular source tree. + + Default: "yes" + + Note that setting MKOBJ to "no" is not recommended and may + cause problems when updating the tree with cvs(1). + + MKPIC Can be set to "yes" or "no". Indicates whether shared + objects and libraries will be created and installed during a + build. If set to "no", the entire built system will be + statically linked. + + Default: Platform dependent. As of this writing, all + platforms except m68000 default to "yes". + + MKPICINSTALL + Can be set to "yes" or "no". Indicates whether the ar(1) + format libraries (lib*_pic.a), used to generate shared + libraries, are installed during a build. + + Default: "yes" + + MKPROFILE Can be set to "yes" or "no". Indicates whether profiled + libraries (lib*_p.a) will be built and installed during a + build. + + Default: "yes"; however, some platforms turn off MKPROFILE by + default at times due to toolchain problems with profiled + code. + + MKREPRO Can be set to "yes" or "no". Create reproducible builds. + This enables different switches to make two builds from the + same source tree result in the same build results. + + Default: "no" This may be set to "yes" by giving build.sh the + -P option. + + MKREPRO_TIMESTAMP + Unix timestamp. When MKREPRO is set, the timestamp of all + files in the sets will be set to this value. + + Default: Unset. This may be set automatically to the latest + source tree timestamp using cvslatest(1) by giving build.sh + the -P option. + + MKSHARE Can be set to "yes" or "no". Indicates whether files + destined to reside in DESTDIR/usr/share will be built and + installed during a build. If set to "no", then all of + MKCATPAGES, MKDOC, MKINFO, MKMAN, and MKNLS will be set to + "no" unconditionally. + + Default: "yes" + + MKSTRIPIDENT + Can be set to "yes" or "no". Indicates whether RCS IDs, for + use with ident(1), should be stripped from program binaries + and shared libraries. + + Default: "no" + + MKSTRIPSYM Can be set to "yes" or "no". Indicates whether all local + symbols should be stripped from shared libraries. If "yes", + strip all local symbols from shared libraries; the affect is + equivalent to the -x option of ld(1). If "no", strip only + temporary local symbols; the affect is equivalent to the -X + option of ld(1). Keeping non-temporary local symbols such as + static function names is useful on using DTrace for userland + libraries and getting a backtrace from a rump kernel loading + shared libraries. + + Default: "yes" + + MKUNPRIVED Can be set to "yes" or "no". Indicates whether an + unprivileged install will occur. The user, group, + permissions, and file flags, will not be set on the installed + items; instead the information will be appended to a file + called METALOG in DESTDIR. The contents of METALOG are used + during the generation of the distribution tar files to ensure + that the appropriate file ownership is stored. + + Default: "no" + + MKUPDATE Can be set to "yes" or "no". Indicates whether all install + operations intended to write to DESTDIR will compare file + timestamps before installing, and skip the install phase if + the destination files are up-to-date. This also has + implications on full builds (see next subsection). + + Default: "no" + + MKX11 Can be set to "yes" or "no". Indicates whether X11 is built + from X11SRCDIR. + + Default: "no" + + TOOLDIR Directory to hold the host tools, once built. If specified, + must be an absolute path. This directory should be unique to + a given host system and NetBSD source tree. (However, + multiple targets may share the same TOOLDIR; the target- + dependent files have unique names.) If unset, a default + based on the uname(1) information of the host platform will + be created in the .OBJDIR of src. + + Default: Unset. + + USETOOLS Indicates whether the tools specified by TOOLDIR should be + used as part of a build in progress. Must be set to "yes" if + cross-compiling. + + yes Use the tools from TOOLDIR. + + no Do not use the tools from TOOLDIR, but refuse to build + native compilation tool components that are version- + specific for that tool. + + never Do not use the tools from TOOLDIR, even when building + native tool components. This is similar to the + traditional NetBSD build method, but does not verify + that the compilation tools in use are up-to-date + enough in order to build the tree successfully. This + may cause build or runtime problems when building the + whole NetBSD source tree. + + Default: "yes", unless TOOLCHAIN_MISSING is set to "yes". + + USETOOLS is also set to "no" when using outside + the NetBSD source tree. + + X11SRCDIR Directory containing the modular Xorg source. If specified, + must be an absolute path. The main modular Xorg source is + found in X11SRCDIR/external/mit. + + Default: NETBSDRCDIR/../xsrc, if that exists; otherwise + /usr/xsrc. + + "make" variables for full builds + These variables only affect the top level "Makefile" and do not affect + manually building subtrees of the NetBSD source code. + + INSTALLWORLDDIR Location for the "make installworld" target to install + to. If specified, must be an absolute path. + + Default: "/" + + MKOBJDIRS Can be set to "yes" or "no". Indicates whether object + directories will be created automatically (via a "make + obj" pass) at the start of a build. + + Default: "no" + + If using build.sh, the default is "yes". This may be + set back to "no" by giving build.sh the -o option. + + MKUPDATE Can be set to "yes" or "no". If set, then in addition + to the effects described for MKUPDATE=yes above, this + implies the effects of NOCLEANDIR (i.e., "make cleandir" + is avoided). + + Default: "no" + + If using build.sh, this may be set by giving the -u + option. + + NBUILDJOBS Now obsolete. Use the make(1) option -j, instead. See + below. + + Default: Unset. + + NOCLEANDIR If set, avoids the "make cleandir" phase of a full + build. This has the effect of allowing only changed + files in a source tree to be recompiled. This can speed + up builds when updating only a few files in the tree. + + Default: Unset. + + See also MKUPDATE. + + NODISTRIBDIRS If set, avoids the "make distrib-dirs" phase of a full + build. This skips running mtree(8) on DESTDIR, useful + on systems where building as an unprivileged user, or + where it is known that the system-wide mtree files have + not changed. + + Default: Unset. + + NOINCLUDES If set, avoids the "make includes" phase of a full + build. This has the effect of preventing make(1) from + thinking that some programs are out-of-date simply + because the system include files have changed. However, + this option should not be used when updating the entire + NetBSD source tree arbitrarily; it is suggested to use + MKUPDATE=yes instead in that case. + + Default: Unset. + + RELEASEDIR If set, specifies the directory to which a release(7) + layout will be written at the end of a "make release". + If specified, must be an absolute path. + + Default: Unset. + + Note: build.sh will provide a default of releasedir (in + the top-level .OBJDIR) unless run in `expert' mode. + +BUILDING + "make" command line options + This is not a summary of all the options available to make(1); only the + options used most frequently with NetBSD builds are listed here. + + -j njob Run up to njob make(1) subjobs in parallel. Makefiles should + use .WAIT or have explicit dependencies as necessary to + enforce build ordering. + + -m dir Specify the default directory for searching for system + Makefile segments, mainly the files. When building + any full NetBSD source tree, this should be set to the + "share/mk" directory in the source tree. This is set + automatically when building from the top level, or when using + build.sh. + + -n Display the commands that would have been executed, but do not + actually execute them. This will still cause recursion to + take place. + + -V var Print make(1)'s idea of the value of var. Does not build any + targets. + + var=value Set the variable var to value, overriding any setting + specified by the process environment, the MAKECONF + configuration file, or the system Makefile segments. + + "make" targets + These default targets may be built by running make(1) in any subtree of + the NetBSD source code. It is recommended that none of these be used + from the top level Makefile; as a specific exception, "make obj" and + "make cleandir" are useful in that context. + + all Build programs, libraries, and preformatted documentation. + + clean Remove program and library object code files. + + cleandir Same as clean, but also remove preformatted documentation, + dependency files generated by "make depend", and any other + files known to be created at build time. + + depend Create dependency files (.depend) containing more detailed + information about the dependencies of source code on header + files. Allows programs to be recompiled automatically when a + dependency changes. + + dependall Does a "make depend" immediately followed by a "make all". + This improves cache locality of the build since both passes + read the source files in their entirety. + + distclean Synonym for cleandir. + + includes Build and install system header files. Typically needed + before any system libraries or programs can be built. + + install Install programs, libraries, and documentation into DESTDIR. + Few files will be installed to DESTDIR/dev, DESTDIR/etc, + DESTDIR/root or DESTDIR/var in order to prevent user supplied + configuration data from being overwritten. + + lint Run lint(1) against the C source code, where appropriate, and + generate system-installed lint libraries. + + obj Create object directories to be used for built files, instead + of building directly in the source tree. + + tags Create ctags(1) searchable function lists usable by the ex(1) + and vi(1) text editors. + + "make" targets for the top level + Additional make(1) targets are usable specifically from the top source + level to facilitate building the entire NetBSD source tree. + + build Build the entire NetBSD system (except the kernel). This + orders portions of the source tree such that prerequisites + will be built in the proper order. + + distribution Do a "make build", and then install a full distribution + (which does not include a kernel) into DESTDIR, including + files in DESTDIR/dev, DESTDIR/etc, DESTDIR/root and + DESTDIR/var. + + buildworld As per "make distribution", except that it ensures that + DESTDIR is not the root directory. + + installworld Install the distribution from DESTDIR to INSTALLWORLDDIR, + which defaults to the root directory. Ensures that + INSTALLWORLDDIR is not the root directory if cross + compiling. + + The INSTALLSETS environment variable may be set to a space- + separated list of distribution sets to be installed. By + default, all sets except "etc" and "xetc" are installed, so + most files in INSTALLWORLDDIR/etc will not be installed or + modified. + + Note: Before performing this operation with + INSTALLWORLDDIR=/, it is highly recommended that you + upgrade your kernel and reboot. After performing this + operation, it is recommended that you use etcupdate(8) to + update files in INSTALLWORLDDIR/etc, and postinstall(8) to + check for or fix inconsistencies. + + sets Create distribution sets from DESTDIR into + RELEASEDIR/RELEASEMACHINEDIR/binary/sets. Should be run + after "make distribution", as "make build" alone does not + install all of the required files. + + sourcesets Create source sets of the source tree into + RELEASEDIR/source/sets. + + syspkgs Create syspkgs from DESTDIR into + RELEASEDIR/RELEASEMACHINEDIR/binary/syspkgs. Should be run + after "make distribution", as "make build" alone does not + install all of the required files. + + release Do a "make distribution", build kernels, distribution + media, and install sets (this as per "make sets"), and then + package the system into a standard release layout as + described by release(7). This requires that RELEASEDIR be + set (see above). + + iso-image Create a NetBSD installation CD-ROM image in the + RELEASEDIR/images directory. The CD-ROM file system will + have a layout as described in release(7). + + For most machine types, the CD-ROM will be bootable, and + will automatically run the sysinst(8) menu-based + installation program, which can be used to install or + upgrade a NetBSD system. Bootable CD-ROMs also contain + tools that may be useful in repairing a damaged NetBSD + installation. + + Before "make iso-image" is attempted, RELEASEDIR must be + populated by "make release" or equivalent. + + Note that other, smaller, CD-ROM images may be created in + the RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom + directory by "make release". These smaller images usually + contain the same tools as the larger images in + RELEASEDIR/images, but do not contain additional content + such as the distribution sets. + + Note that the mac68k port still uses an older method of + creating CD-ROM images. This requires the mkisofs(1) + utility, which is not part of NetBSD, but which can be + installed from pkgsrc/sysutils/cdrtools. + + iso-image-source + Create a NetBSD installation CD-ROM image in the + RELEASEDIR/images directory. The CD-ROM file system will + have a layout as described in release(7). It will have top + level directories for the machine type and source. + + For most machine types, the CD-ROM will be bootable, and + will automatically run the sysinst(8) menu-based + installation program, which can be used to install or + upgrade a NetBSD system. Bootable CD-ROMs also contain + tools that may be useful in repairing a damaged NetBSD + installation. + + Before "make iso-image-source" is attempted, RELEASEDIR + must be populated by "make sourcesets release" or + equivalent. + + Note that other, smaller, CD-ROM images may be created in + the RELEASEDIR/RELEASEMACHINEDIR/installation/cdrom + directory by "make release". These smaller images usually + contain the same tools as the larger images in + RELEASEDIR/images, but do not contain additional content + such as the distribution sets. + + Note that the mac68k port still uses an older method of + creating CD-ROM images. This requires the mkisofs(1) + utility, which is not part of NetBSD, but which can be + installed from pkgsrc/sysutils/cdrtools. + + install-image + Create a bootable NetBSD installation disk image in the + RELEASEDIR/images directory. The installation disk image + is suitable for copying to bootable USB flash memory + sticks, etc., for machines which are able to boot from such + devices. The file system in the bootable disk image will + have a layout as described in release(7). + + The installation image is bootable, and will automatically + run the sysinst(8) menu-based installation program, which + can be used to install or upgrade a NetBSD system. The + image also contains tools that may be useful in repairing a + damaged NetBSD installation. + + Before "make install-image" is attempted, RELEASEDIR must + be populated by "make release" or equivalent. The build + must have been performed with MKUNPRIVED=yes because "make + install-image" relies on information in DESTDIR/METALOG. + + live-image Create NetBSD live images in the RELEASEDIR/images + directory. The live image contains all necessary files to + boot NetBSD up to multi-user mode, including all files + which should be extracted during installation, NetBSD + disklabel, bootloaders, etc. + + The live image is suitable for use as a disk image in + virtual machine environments such as QEMU, and also useful + to boot NetBSD from a USB flash memory stick on a real + machine, without the need for installation. + + Before "make live-image" is attempted, RELEASEDIR must be + populated by "make release" or equivalent. The build must + have been performed with MKUNPRIVED=yes because "make + install-image" relies on information in DESTDIR/METALOG. + + regression-tests + Can only be run after building the regression tests in the + directory "regress". Runs those compiled regression tests + on the local host. Note that most tests are now managed + instead using atf(7); this target should probably run those + as well but currently does not. + + The "build.sh" script + This script file is a shell script designed to build the entire NetBSD + system on any host with a suitable modern shell and some common + utilities. The required shell features are described under the HOST_SH + variable. + + If a host system's default shell does support the required features, then + we suggest that you explicitly specify a suitable shell using a command + like + + /path/to/suitable/shell build.sh [options] + + The above command will usually enable build.sh to automatically set + HOST_SH=/path/to/suitable/shell, but if that fails, then the following + set of commands may be used instead: + + HOST_SH=/path/to/suitable/shell + export HOST_SH + ${HOST_SH} build.sh [options] + + If build.sh detects that it is being executed under an unsuitable shell, + it attempts to exec a suitable shell instead, or prints an error message. + If HOST_SH is not set explicitly, then build.sh sets a default using + heuristics dependent on the host platform, or from the shell under which + build.sh is executed (if that can be determined), or using the first copy + of sh found in PATH. + + All cross-compile builds, and most native builds, of the entire system + should make use of build.sh rather than just running "make". This way, + the make(1) program will be bootstrapped properly, in case the host + system has an older or incompatible "make" program. + + When compiling the entire system via build.sh, many make(1) variables are + set for you in order to help encapsulate the build process. In the list + of options below, variables that are automatically set by build.sh are + noted where applicable. + + The following operations are supported by build.sh: + + build Build the system as per "make build". Before the main part + of the build commences, this command runs the obj operation + (unless the -o option is given), "make cleandir" (unless + the -u option is given), and the tools operation. + + distribution Build a full distribution as per "make distribution". This + command first runs the build operation. + + release Build a full release as per "make release". This command + first runs the distribution operation. + + makewrapper Create the nbmake-MACHINE wrapper. This operation is + automatically performed for any of the other operations. + + cleandir Perform "make cleandir". + + obj Perform "make obj". + + tools Build and install the host tools from src/tools. This + command will first run "make obj" and "make cleandir" in + the tools subdirectory unless the -o or -u options + (respectively) are given. + + install=idir Install the contents of DESTDIR to idir, using "make + installworld". Note that files that are part of the "etc" + or "xetc" sets will not be installed, unless overridden by + the INSTALLSETS environment variable. + + kernel=kconf Build a new kernel. The kconf argument is the name of a + configuration file suitable for use by config(1). If kconf + does not contain any `/' characters, the configuration file + is expected to be found in the KERNCONFDIR directory, which + is typically sys/arch/MACHINE/conf. The new kernel will be + built in a subdirectory of KERNOBJDIR, which is typically + sys/arch/MACHINE/compile or an associated object directory. + + This command does not imply the tools command; run the + tools command first unless it is certain that the tools + already exist and are up to date. + + This command will run "make cleandir" on the kernel in + question first unless the -u option is given. + + kernel.gdb=kconf + Build a new kernel with debug information. Similar to the + above kernel=kconf operation, but creates a netbsd.gdb file + alongside of the kernel netbsd, which contains a full + symbol table and can be used for debugging (for example + with a cross-gdb built by MKCROSSGDB). + + kernels This command will build all kernels defined in port + specific release build procedure. + + This command internally calls the kernel=kconf operation + for each found kernel configuration file. + + modules This command will build kernel modules and install them + into DESTDIR. + + releasekernel=kconf + Install a gzip(1)ed copy of the kernel previously built by + kernel=kconf into + RELEASEDIR/RELEASEMACHINEDIR/binary/kernel, usually as + netbsd-kconf.gz, although the "netbsd" prefix is determined + from the "config" directives in kconf. + + sets Perform "make sets". + + sourcesets Perform "make sourcesets". + + syspkgs Perform "make syspkgs". + + iso-image Perform "make iso-image". + + iso-image-source + Perform "make iso-image-source". + + install-image + Perform "make install-image". + + live-image Perform "make live-image". + + list-arch Prints a list of valid MACHINE and MACHINE_ARCH settings, + the default MACHINE_ARCH for each MACHINE, and aliases for + MACHINE/MACHINE_ARCH pairs, and then exits. The -m or -a + options (or both) may be used to specify glob patterns that + will be used to narrow the list of results; for example, + "build.sh -m 'evb*' -a '*arm*' list-arch" will list all + known MACHINE/MACHINE_ARCH values in which either MACHINE + or ALIAS matches the pattern `evb*', and MACHINE_ARCH + matches the pattern `*arm*'. + + The following command line options alter the behaviour of the build.sh + operations described above: + + -a arch Set the value of MACHINE_ARCH to arch. See the -m option for + more information. + + -B buildid + Set the value of BUILDID to buildid. This will also append the + build identifier to the name of the "make" wrapper script so + that the resulting name is of the form + "nbmake-MACHINE-BUILDID". + + -C cdextras + Append cdextras to the CDEXTRA variable, which is a space- + separated list of files or directories that will be added to + the CD-ROM image that may be create by the "iso-image" or + "iso-image-source" operations. Files will be added to the root + of the CD-ROM image, whereas directories will be copied + recursively. If relative paths are specified, they will be + converted to absolute paths before being used. Multiple paths + may be specified via multiple -C options, or via a single + option whose argument contains multiple space-separated paths. + + -D dest Set the value of DESTDIR to dest. If a relative path is + specified, it will be converted to an absolute path before + being used. + + -E Set `expert' mode. This overrides various sanity checks, and + allows: DESTDIR does not have to be set to a non-root path for + builds, and MKUNPRIVED=yes does not have to be set when + building as a non-root user. + + Note: It is highly recommended that you know what you are doing + when you use this option. + + -h Print a help message. + + -j njob Run up to njob make(1) subjobs in parallel; passed through to + make(1). If you see failures for reasons other than running + out of memory while using build.sh with -j, please save + complete build logs so the failures can be analyzed. + + To achieve the fastest builds, -j values between (1 + the + number of CPUs) and (2 * the number of CPUs) are recommended. + Use lower values on machines with limited memory or I/O + bandwidth. + + -M obj Set MAKEOBJDIRPREFIX to obj. Unsets MAKEOBJDIR. See "-O obj" + for more information. + + For instance, if the source directory is /usr/src, a setting of + "-M /usr/obj" will place build-time files under + /usr/obj/usr/src/bin, /usr/obj/usr/src/lib, + /usr/obj/usr/src/usr.bin, and so forth. + + If a relative path is specified, it will be converted to an + absolute path before being used. build.sh imposes the + restriction that the argument to the -M option must not begin + with a "$" (dollar sign) character; otherwise it would be too + difficult to determine whether the value is an absolute or a + relative path. If the directory does not already exist, + build.sh will create it. + + -m mach Set the value of MACHINE to mach, unless the mach argument is + an alias that refers to a MACHINE/MACHINE_ARCH pair, in which + case both MACHINE and MACHINE_ARCH are set from the alias. + Such aliases are interpreted entirely by build.sh; they are not + used by any other part of the build system. The MACHINE_ARCH + setting implied by mach will override any value of MACHINE_ARCH + in the process environment, but will not override a value set + by the -a option. All cross builds require -m, but if unset on + a NetBSD host, the host's value of MACHINE will be detected and + used automatically. + + See the list-arch operation for a way to get a list of valid + MACHINE and MACHINE_ARCH settings. + + -N noiselevel + Set the "noisyness" level of the build, by setting MAKEVERBOSE + to noiselevel. + + -n Show the commands that would be executed by build.sh, but do + not make any changes. This is similar in concept to "make -n". + + -O obj Create an appropriate transform macro for MAKEOBJDIR that will + place the built object files under obj. Unsets + MAKEOBJDIRPREFIX. + + For instance, a setting of "-O /usr/obj" will place build-time + files under /usr/obj/bin, /usr/obj/lib, /usr/obj/usr.bin, and + so forth. + + If a relative path is specified, it will be converted to an + absolute path before being used. build.sh imposes the + restriction that the argument to the -O option must not contain + a "$" (dollar sign) character. If the directory does not + already exist, build.sh will create it. + + In normal use, exactly one of the -M or -O options should be + specified. If neither -M nor -O is specified, then a default + object directory will be chosen according to rules in + . Relying on this default is not recommended + because it is determined by complex rules that are influenced + by the values of several variables and by the location of the + source directory. + + Note that placing the obj directory location outside of the + default source tree hierarchy makes it easier to manually clear + out old files in the event the "make cleandir" operation is + unable to do so. (See CAVEATS below.) + + Note also that use of one of -M or -O is the only means of + building multiple machine architecture userlands from the same + source tree without cleaning between builds (in which case, one + would specify distinct obj locations for each). + + -o Set the value of MKOBJDIRS to "no". Otherwise, it will be + automatically set to "yes". This default is opposite to the + behaviour when not using build.sh. + + -R rel Set the value of RELEASEDIR to rel. If a relative path is + specified, it will be converted to an absolute path before + being used. + + -r Remove the contents of DESTDIR and TOOLDIR before building + (provides a clean starting point). This will skip deleting + DESTDIR if building on a native system to the root directory. + + -S seed Change the value of BUILDSEED to seed. This should rarely be + necessary. + + -T tools Set the value of TOOLDIR to tools. If a relative path is + specified, it will be converted to an absolute path before + being used. If set, the bootstrap "make" will only be rebuilt + if the source files for make(1) have changed. + + -U Set MKUNPRIVED=yes. + + -u Set MKUPDATE=yes. + + -V var=[value] + Set the environment variable var to an optional value. This is + propagated to the nbmake wrapper. + + -w wrapper + Create the nbmake wrapper script (see below) in a custom + location, specified by wrapper. This allows, for instance, to + place the wrapper in PATH automatically. Note that wrapper is + the full name of the file, not just a directory name. If a + relative path is specified, it will be converted to an absolute + path before being used. + + -X x11src + Set the value of X11SRCDIR to x11src. If a relative path is + specified, it will be converted to an absolute path before + being used. + + -x Set MKX11=yes. + + -Y extsrcdir + Set the value of EXTSRCSRCDIR to extsrcdir. If a relative path + is specified, it will be converted to an absolute path before + being used. + + -y Set MKEXTSRC=yes. + + -Z var Unset ("zap") the environment variable var. This is propagated + to the nbmake wrapper. + + The "nbmake-MACHINE" wrapper script + If using the build.sh script to build NetBSD, a nbmake-MACHINE script + will be created in TOOLDIR/bin upon the first build to assist in building + subtrees on a cross-compile host. + + nbmake-MACHINE can be invoked in lieu of make(1), and will instead call + the up-to-date version of "nbmake" installed into TOOLDIR/bin with + several key variables pre-set, including MACHINE, MACHINE_ARCH, and + TOOLDIR. nbmake-MACHINE will also set variables specified with -V, and + unset variables specified with -Z. + + This script can be symlinked into a directory listed in PATH, or called + with an absolute path. + +EXAMPLES + 1. % ./build.sh [options] tools kernel=GENERIC + + Build a new toolchain, and use the new toolchain to configure and + build a new GENERIC kernel. + + 2. % ./build.sh [options] -U distribution + + Using unprivileged mode, build a complete distribution to a DESTDIR + directory that build.sh selects (and will display). + + 3. # ./build.sh [options] -U install=/ + + As root, install to / the distribution that was built by example 2. + Even though this is run as root, -U is required so that the + permissions stored in DESTDIR/METALOG are correctly applied to the + files as they're copied to /. + + 4. % ./build.sh [options] -U -u release + + Using unprivileged mode, build a complete release to DESTDIR and + RELEASEDIR directories that build.sh selects (and will display). + MKUPDATE=yes (-u) is set to prevent the "make cleandir", so that if + this is run after example 2, it doesn't need to redo that portion of + the release build. + +OBSOLETE VARIABLES + NBUILDJOBS Use the make(1) option -j instead. + + USE_NEW_TOOLCHAIN + The new toolchain is now the default. To disable, use + TOOLCHAIN_MISSING=yes. + +SEE ALSO + make(1), hier(7), release(7), etcupdate(8), postinstall(8), sysinst(8), + pkgsrc/sysutils/cdrtools + +HISTORY + The build.sh based build scheme was introduced for NetBSD 1.6 as + USE_NEW_TOOLCHAIN, and re-worked to TOOLCHAIN_MISSING after that. + +CAVEATS + After significant updates to third-party components in the source tree, + the "make cleandir" operation may be insufficient to clean out old files + in object directories. Instead, one may have to manually remove the + files. Consult the UPDATING file for notices concerning this. + +NetBSD October 21, 2017 NetBSD diff --git a/Makefile b/Makefile new file mode 100644 index 000000000000..c068f130dc6f --- /dev/null +++ b/Makefile @@ -0,0 +1,531 @@ +# $NetBSD: Makefile,v 1.324 2018/05/02 07:34:44 pgoyette Exp $ + +# +# This is the top-level makefile for building NetBSD. For an outline of +# how to build a snapshot or release, as well as other release engineering +# information, see http://www.NetBSD.org/developers/releng/index.html +# +# Not everything you can set or do is documented in this makefile. In +# particular, you should review the files in /usr/share/mk (especially +# bsd.README) for general information on building programs and writing +# Makefiles within this structure, and see the comments in src/etc/Makefile +# for further information on installation and release set options. +# +# Variables listed below can be set on the make command line (highest +# priority), in /etc/mk.conf (middle priority), or in the environment +# (lowest priority). +# +# Variables: +# DESTDIR is the target directory for installation of the compiled +# software. It defaults to /. Note that programs are built against +# libraries installed in DESTDIR. +# MKMAN, if `no', will prevent building of manual pages. +# MKOBJDIRS, if not `no', will build object directories at +# an appropriate point in a build. +# MKSHARE, if `no', will prevent building and installing +# anything in /usr/share. +# MKUPDATE, if not `no', will avoid a `make cleandir' at the start of +# `make build', as well as having the effects listed in +# /usr/share/mk/bsd.README. +# NOCLEANDIR, if defined, will avoid a `make cleandir' at the start +# of the `make build'. +# NOINCLUDES will avoid the `make includes' usually done by `make build'. +# NOBINARIES will not build binaries, only includes and libraries +# +# See mk.conf(5) for more details. +# +# +# Targets: +# build: +# Builds a full release of NetBSD in DESTDIR, except for the +# /etc configuration files. +# If BUILD_DONE is set, this is an empty target. +# distribution: +# Builds a full release of NetBSD in DESTDIR, including the /etc +# configuration files. +# buildworld: +# As per `make distribution', except that it ensures that DESTDIR +# is not the root directory. +# installworld: +# Install the distribution from DESTDIR to INSTALLWORLDDIR (which +# defaults to the root directory). Ensures that INSTALLWORLDDIR +# is not the root directory if cross compiling. +# release: +# Does a `make distribution', and then tars up the DESTDIR files +# into ${RELEASEDIR}/${RELEASEMACHINEDIR}, in release(7) format. +# (See etc/Makefile for more information on this.) +# regression-tests: +# Runs the regression tests in "regress" on this host. +# sets: +# Populate ${RELEASEDIR}/${RELEASEMACHINEDIR}/binary/sets +# from ${DESTDIR} +# sourcesets: +# Populate ${RELEASEDIR}/source/sets from ${NETBSDSRCDIR} +# syspkgs: +# Populate ${RELEASEDIR}/${RELEASEMACHINEDIR}/binary/syspkgs +# from ${DESTDIR} +# iso-image: +# Create CD-ROM image in RELEASEDIR/images. +# RELEASEDIR must already have been populated by `make release' +# or equivalent. +# iso-image-source: +# Create CD-ROM image with source in RELEASEDIR/images. +# RELEASEDIR must already have been populated by +# `make release sourcesets' or equivalent. +# live-image: +# Create bootable live image for emulators or USB stick etc. +# in RELEASEDIR/liveimage. +# RELEASEDIR must already have been populated by `make release' +# or equivalent. +# install-image: +# Create bootable installation image for USB stick etc. +# in RELEASEDIR/installimage. +# RELEASEDIR must already have been populated by `make release' +# or equivalent. +# +# Targets invoked by `make build,' in order: +# cleandir: cleans the tree. +# do-top-obj: creates the top level object directory. +# do-tools-obj: creates object directories for the host toolchain. +# do-tools: builds host toolchain. +# params: record the values of variables that might affect the +# build. +# obj: creates object directories. +# do-distrib-dirs: creates the distribution directories. +# includes: installs include files. +# do-lib: builds and installs prerequisites from lib +# if ${MKCOMPAT} != "no". +# do-compat-lib: builds and installs prerequisites from compat/lib +# if ${MKCOMPAT} != "no". +# do-x11: builds and installs X11 tools and libraries +# from src/external/mit/xorg if ${MKX11} != "no". +# do-build: builds and installs the entire system. +# do-extsrc: builds and installs extsrc if ${MKEXTSRC} != "no". +# do-obsolete: installs the obsolete sets (for the postinstall-* targets). +# + +.if ${.MAKEFLAGS:M${.CURDIR}/share/mk} == "" +.MAKEFLAGS: -m ${.CURDIR}/share/mk +.endif + +# +# If _SRC_TOP_OBJ_ gets set here, we will end up with a directory that may +# not be the top level objdir, because "make obj" can happen in the *middle* +# of "make build" (long after is calculated it). So, pre-set +# _SRC_TOP_OBJ_ here so it will not be added to ${.MAKEOVERRIDES}. +# +_SRC_TOP_OBJ_= + +.include + +# +# Sanity check: make sure that "make build" is not invoked simultaneously +# with a standard recursive target. +# + +.if make(build) || make(release) || make(snapshot) +.for targ in ${TARGETS:Nobj:Ncleandir} +.if make(${targ}) && !target(.BEGIN) +.BEGIN: + @echo 'BUILD ABORTED: "make build" and "make ${targ}" are mutually exclusive.' + @false +.endif +.endfor +.endif + +# +# _SUBDIR is used to set SUBDIR, after removing directories that have +# BUILD_${dir}=no, or that have no ${dir}/Makefile. +# +_SUBDIR= tools lib include external crypto/external bin games +_SUBDIR+= libexec sbin usr.bin +_SUBDIR+= usr.sbin share sys etc tests compat +_SUBDIR+= .WAIT rescue .WAIT distrib regress + +.for dir in ${_SUBDIR} +.if "${dir}" == ".WAIT" \ + || (${BUILD_${dir}:Uyes} != "no" && exists(${dir}/Makefile)) +SUBDIR+= ${dir} +.endif +.endfor + +.if exists(regress) +regression-tests: .PHONY .MAKE + @echo Running regression tests... + ${MAKEDIRTARGET} regress regress +.endif + +.if ${MKUNPRIVED} != "no" +NOPOSTINSTALL= # defined +.endif + +afterinstall: .PHONY .MAKE +.if ${MKMAN} != "no" + ${MAKEDIRTARGET} share/man makedb +.endif +.if (${MKUNPRIVED} != "no" && ${MKINFO} != "no") + ${MAKEDIRTARGET} external/gpl2/texinfo/bin/install-info infodir-meta +.endif +.if !defined(NOPOSTINSTALL) + ${MAKEDIRTARGET} . postinstall-check +.endif + +_POSTINSTALL= ${.CURDIR}/usr.sbin/postinstall/postinstall \ + -m ${MACHINE} -a ${MACHINE_ARCH} +_POSTINSTALL_ENV= \ + AWK=${TOOL_AWK:Q} \ + DB=${TOOL_DB:Q} \ + HOST_SH=${HOST_SH:Q} \ + MAKE=${MAKE:Q} \ + PWD_MKDB=${TOOL_PWD_MKDB:Q} \ + SED=${TOOL_SED:Q} \ + STAT=${TOOL_STAT:Q} + +.if ${MKX11} != "no" +_POSTINSTALL_X11=-x ${X11SRCDIR:Q} +.endif + +postinstall-check: .PHONY + @echo " === Post installation checks ===" + ${_POSTINSTALL_ENV} ${HOST_SH} ${_POSTINSTALL} -s ${.CURDIR} ${_POSTINSTALL_X11} -d ${DESTDIR}/ check; if [ $$? -gt 1 ]; then exit 1; fi + @echo " ================================" + +postinstall-fix: .NOTMAIN .PHONY + @echo " === Post installation fixes ===" + ${_POSTINSTALL_ENV} ${HOST_SH} ${_POSTINSTALL} -s ${.CURDIR} ${_POSTINSTALL_X11} -d ${DESTDIR}/ fix + @echo " ===============================" + +postinstall-fix-obsolete: .NOTMAIN .PHONY + @echo " === Removing obsolete files ===" + ${_POSTINSTALL_ENV} ${HOST_SH} ${_POSTINSTALL} -s ${.CURDIR} ${_POSTINSTALL_X11} -d ${DESTDIR}/ fix obsolete + @echo " ===============================" + +postinstall-fix-obsolete_stand: .NOTMAIN .PHONY + @echo " === Removing obsolete files ===" + ${_POSTINSTALL_ENV} ${HOST_SH} ${_POSTINSTALL} -s ${.CURDIR} ${_POSTINSTALL_X11} -d ${DESTDIR}/ fix obsolete_stand + @echo " ===============================" + + +# +# Targets (in order!) called by "make build". +# +BUILDTARGETS+= check-tools +.if ${MKUPDATE} == "no" && !defined(NOCLEANDIR) +BUILDTARGETS+= cleandir +.endif +.if ${MKOBJDIRS} != "no" +BUILDTARGETS+= do-top-obj +.endif +.if ${USETOOLS} == "yes" # { +.if ${MKOBJDIRS} != "no" +BUILDTARGETS+= do-tools-obj +.endif +BUILDTARGETS+= do-tools +.endif # USETOOLS # } +BUILDTARGETS+= params +.if ${MKOBJDIRS} != "no" +BUILDTARGETS+= obj +.endif +BUILDTARGETS+= clean_METALOG +.if !defined(NODISTRIBDIRS) +BUILDTARGETS+= do-distrib-dirs +.endif +.if !defined(NOINCLUDES) +BUILDTARGETS+= includes +.endif +BUILDTARGETS+= do-lib +BUILDTARGETS+= do-compat-lib +.if ${MKX11} != "no" +BUILDTARGETS+= do-x11 +.endif +.if !defined(NOBINARIES) +BUILDTARGETS+= do-build +.if ${MKEXTSRC} != "no" +BUILDTARGETS+= do-extsrc +.endif +BUILDTARGETS+= do-obsolete +.endif + +# +# Enforce proper ordering of some rules. +# + +.ORDER: ${BUILDTARGETS} +includes-lib: .PHONY includes-include includes-sys + +# +# Record the values of variables that might affect the build. +# If no values have changed, avoid updating the timestamp +# of the params file. +# +# This is referenced by _NETBSD_VERSION_DEPENDS in . +# +.include "${NETBSDSRCDIR}/etc/Makefile.params" +CLEANDIRFILES+= params +params: .EXEC + ${_MKMSG_CREATE} params + @${PRINT_PARAMS} >${.TARGET}.new + @if cmp -s ${.TARGET}.new ${.TARGET} > /dev/null 2>&1; then \ + : "params is unchanged" ; \ + rm ${.TARGET}.new ; \ + else \ + : "params has changed or is new" ; \ + mv ${.TARGET}.new ${.TARGET} ; \ + fi + +# +# Display current make(1) parameters +# +show-params: .PHONY .MAKE + @${PRINT_PARAMS} + +# +# Build the system and install into DESTDIR. +# + +START_TIME!= date + +build: .PHONY .MAKE +.if defined(BUILD_DONE) + @echo "Build already installed into ${DESTDIR}" +.else + @echo "Build started at: ${START_TIME}" +.for tgt in ${BUILDTARGETS} + ${MAKEDIRTARGET} . ${tgt} +.endfor + ${MAKEDIRTARGET} etc install-etc-release + @echo "Build started at: ${START_TIME}" + @printf "Build finished at: " && date +.endif + +# +# Build a full distribution, but not a release (i.e. no sets into +# ${RELEASEDIR}). "buildworld" enforces a build to ${DESTDIR} != / +# + +distribution buildworld: .PHONY .MAKE +.if make(buildworld) && \ + (!defined(DESTDIR) || ${DESTDIR} == "" || ${DESTDIR} == "/") + @echo "Won't make ${.TARGET} with DESTDIR=/" + @false +.endif + ${MAKEDIRTARGET} . build NOPOSTINSTALL=1 + ${MAKEDIRTARGET} etc distribution INSTALL_DONE=1 +.if defined(DESTDIR) && ${DESTDIR} != "" && ${DESTDIR} != "/" + ${MAKEDIRTARGET} . postinstall-fix-obsolete + ${MAKEDIRTARGET} . postinstall-fix-obsolete_stand + ${MAKEDIRTARGET} distrib/sets checkflist +.endif + @echo "make ${.TARGET} started at: ${START_TIME}" + @printf "make ${.TARGET} finished at: " && date + +# +# Install the distribution from $DESTDIR to $INSTALLWORLDDIR (defaults to `/') +# If installing to /, ensures that the host's operating system is NetBSD and +# the host's `uname -m` == ${MACHINE}. +# + +HOST_UNAME_S!= uname -s +HOST_UNAME_M!= uname -m + +installworld: .PHONY .MAKE +.if (!defined(DESTDIR) || ${DESTDIR} == "" || ${DESTDIR} == "/") + @echo "Can't make ${.TARGET} to DESTDIR=/" + @false +.endif +.if !defined(INSTALLWORLDDIR) || \ + ${INSTALLWORLDDIR} == "" || ${INSTALLWORLDDIR} == "/" +.if (${HOST_UNAME_S} != "NetBSD") + @echo "Won't cross-make ${.TARGET} from ${HOST_UNAME_S} to NetBSD with INSTALLWORLDDIR=/" + @false +.endif +.if (${HOST_UNAME_M} != ${MACHINE}) + @echo "Won't cross-make ${.TARGET} from ${HOST_UNAME_M} to ${MACHINE} with INSTALLWORLDDIR=/" + @false +.endif +.endif + ${MAKEDIRTARGET} distrib/sets installsets \ + INSTALLDIR=${INSTALLWORLDDIR:U/} INSTALLSETS=${INSTALLSETS:Q} + ${MAKEDIRTARGET} . postinstall-check DESTDIR=${INSTALLWORLDDIR} + @echo "make ${.TARGET} started at: ${START_TIME}" + @printf "make ${.TARGET} finished at: " && date + +# +# Install modules from $DESTDIR to $INSTALLMODULESDIR +# +installmodules: .PHONY .MAKE +.if (!defined(DESTDIR) || ${DESTDIR} == "" || ${DESTDIR} == "/") + @echo "Can't make ${.TARGET} to DESTDIR=/" + @false +.endif +.if !defined(INSTALLMODULESDIR) || \ + ${INSTALLMODULESDIR} == "" || ${INSTALLMODULESDIR} == "/" +.if (${HOST_UNAME_S} != "NetBSD") + @echo "Won't cross-make ${.TARGET} from ${HOST_UNAME_S} to NetBSD with INSTALLMODULESDIR=/" + @false +.endif +.if (${HOST_UNAME_M} != ${MACHINE}) + @echo "Won't cross-make ${.TARGET} from ${HOST_UNAME_M} to ${MACHINE} with INSTALLMODULESDIR=/" + @false +.endif +.endif + ${MAKEDIRTARGET} sys/modules install DESTDIR=${INSTALLMODULESDIR:U/} + @echo "make ${.TARGET} started at: ${START_TIME}" + @printf "make ${.TARGET} finished at: " && date + +# +# Create sets from $DESTDIR or $NETBSDSRCDIR into $RELEASEDIR +# + +.for tgt in sets sourcesets syspkgs +${tgt}: .PHONY .MAKE + ${MAKEDIRTARGET} distrib/sets ${tgt} +.endfor + +# +# Build a release or snapshot (implies "make distribution"). Note that +# in this case, the set lists will be checked before the tar files +# are made. +# + +release snapshot: .PHONY .MAKE + ${MAKEDIRTARGET} . distribution + ${MAKEDIRTARGET} etc release DISTRIBUTION_DONE=1 + @echo "make ${.TARGET} started at: ${START_TIME}" + @printf "make ${.TARGET} finished at: " && date + +# +# Create a CD-ROM image. +# + +iso-image: .PHONY + ${MAKEDIRTARGET} distrib iso_image + ${MAKEDIRTARGET} etc iso-image + @echo "make ${.TARGET} started at: ${START_TIME}" + @printf "make ${.TARGET} finished at: " && date + +iso-image-source: .PHONY + ${MAKEDIRTARGET} distrib iso_image CDSOURCE=true + ${MAKEDIRTARGET} etc iso-image + @echo "make ${.TARGET} started at: ${START_TIME}" + @printf "make ${.TARGET} finished at: " && date + +# +# Create bootable live images. +# + +live-image: .PHONY + ${MAKEDIRTARGET} etc live-image + @echo "make ${.TARGET} started at: ${START_TIME}" + @printf "make ${.TARGET} finished at: " && date + +# +# Create bootable installation images. +# + +install-image: .PHONY + ${MAKEDIRTARGET} etc install-image + @echo "make ${.TARGET} started at: ${START_TIME}" + @printf "make ${.TARGET} finished at: " && date + +# +# Special components of the "make build" process. +# + +check-tools: .PHONY +.if ${TOOLCHAIN_MISSING} != "no" && !defined(EXTERNAL_TOOLCHAIN) + @echo '*** WARNING: Building on MACHINE=${MACHINE} with missing toolchain.' + @echo '*** May result in a failed build or corrupt binaries!' +.elif defined(EXTERNAL_TOOLCHAIN) + @echo '*** Using external toolchain rooted at ${EXTERNAL_TOOLCHAIN}.' +.endif +.if defined(NBUILDJOBS) + @echo '*** WARNING: NBUILDJOBS is obsolete; use -j directly instead!' +.endif + +# Delete or sanitise a leftover METALOG from a previous build. +clean_METALOG: .PHONY .MAKE +.if ${MKUPDATE} != "no" + ${MAKEDIRTARGET} distrib/sets clean_METALOG +.endif + +do-distrib-dirs: .PHONY .MAKE +.if !defined(DESTDIR) || ${DESTDIR} == "" + ${MAKEDIRTARGET} etc distrib-dirs DESTDIR=/ +.else + ${MAKEDIRTARGET} etc distrib-dirs DESTDIR=${DESTDIR} +.endif + +.for targ in cleandir obj includes +do-${targ}: .PHONY ${targ} + @true +.endfor + +do-tools: .PHONY .MAKE + ${MAKEDIRTARGET} tools build_install + +do-lib: .PHONY .MAKE + ${MAKEDIRTARGET} lib build_install + +do-compat-lib: .PHONY .MAKE + ${MAKEDIRTARGET} compat build_install BOOTSTRAP_SUBDIRS="../../../lib" + +do-top-obj: .PHONY .MAKE + ${MAKEDIRTARGET} . obj NOSUBDIR= + +do-tools-obj: .PHONY .MAKE + ${MAKEDIRTARGET} tools obj + +do-build: .PHONY .MAKE +.for targ in dependall install + ${MAKEDIRTARGET} . ${targ} BUILD_tools=no BUILD_lib=no +.endfor + +do-x11: .PHONY .MAKE +.if ${MKX11} != "no" + ${MAKEDIRTARGET} external/mit/xorg/tools all + ${MAKEDIRTARGET} external/mit/xorg/lib build_install +.if ${MKCOMPATX11} != "no" + ${MAKEDIRTARGET} compat build_install BOOTSTRAP_SUBDIRS="../../../external/mit/xorg/lib" +.endif +.else + @echo "MKX11 is not enabled" + @false +.endif + +do-extsrc: .PHONY .MAKE +.if ${MKEXTSRC} != "no" + ${MAKEDIRTARGET} extsrc build +.else + @echo "MKEXTSRC is not enabled" + @false +.endif + +do-obsolete: .PHONY .MAKE + ${MAKEDIRTARGET} etc install-obsolete-lists + +# +# Speedup stubs for some subtrees that don't need to run these rules. +# (Tells not to recurse for them.) +# + +.for dir in bin etc distrib games libexec regress sbin usr.bin usr.sbin tools +includes-${dir}: .PHONY + @true +.endfor +.for dir in etc distrib regress +install-${dir}: .PHONY + @true +.endfor + +# +# XXX this needs to change when distrib Makefiles are recursion compliant +# XXX many distrib subdirs need "cd etc && make snap_pre snap_kern" first... +# +dependall-distrib depend-distrib all-distrib: .PHONY + @true + +.include +.include +.include +.include diff --git a/Makefile.inc b/Makefile.inc new file mode 100644 index 000000000000..f8fd5cec3d82 --- /dev/null +++ b/Makefile.inc @@ -0,0 +1,16 @@ +# $NetBSD: Makefile.inc,v 1.6 2018/05/02 07:34:44 pgoyette Exp $ + +.ifndef ABSTOP +ABSTOP!= cd ${.PARSEDIR}; pwd + +.if (${.PARSEDIR} == ${.CURDIR}) +RELTOP:= . +.else +RELTOP:= ${.PARSEDIR:S,^${.CURDIR},,} +.endif + +# you can set BSD{SRC,OBJ}DIR in Makefile.local.inc +.if exists(${ABSTOP}/Makefile.local.inc) +.include "${ABSTOP}/Makefile.local.inc" +.endif +.endif diff --git a/UPDATING b/UPDATING new file mode 100644 index 000000000000..bdd7dacad8a2 --- /dev/null +++ b/UPDATING @@ -0,0 +1,852 @@ +$NetBSD: UPDATING,v 1.295 2018/05/02 07:34:44 pgoyette Exp $ + +This file (UPDATING) is intended to be a brief reference to recent +changes that might cause problems in the build process, and a guide for +what to do if something doesn't work. + +For a more detailed description of the recommended way to build NetBSD +using build.sh, see the BUILDING file. + +Note that much of the advice in this UPDATING file was written before +build.sh existed. Nevertheless, the advice here may be useful for +working around specific problems with build.sh. + +Sections are marked with "^^^^^". After the section on "Recent changes" +are several sections containing more general information. + +See also: BUILDING, build.sh, Makefile. + +Recent changes: +^^^^^^^^^^^^^^^ + +20180414: + Existing binutils was migrated to binutils.old. Manual + removal of tools/binutils objects directory may be required + to fix tools build failure. + +20180311: + bdftopcf was updated and may need cleaning in the + src/external/mit/xorg/tools/bdftopcf subdirectory if there are + link errors. + +20180212: + between OpenSSL and GCC updates, many things may fail to build. + any failure that looks like GCC or openssl is best handled by + a clean destdir and objdir. Full cleandir and destdir deletion + is recommended if build failures occur. + +20171225: + removal of the vadvise syscall requires manual removal of all + associated files from the libc build object directory (including + the .depend files) - a command like: + cd $OBJ && find . -type d -name libc | xargs rm -rf + For architectures that support multiple "compat" binary targets, + you'll need to cleanup both the regular libc directory and the + compat ones (the above command will do that). + +20171010: + a change to the build structure of external/bsd/acpica/bin/iasl + means that its objdir (or *.d and .depend at least) might need + to be manually removed - or a build done once without -u. + +20170822: + a new version of GMP has been imported and probably + will break parts of builds related to themselves or GCC, both + in the tools and the native section. Remove all GCC, GMP, MPFR + and MPC objdirs or build once without -u. + +20170816: + a new version of MPFR and MPC have been imported and probably + will break parts of builds related to themselves or GCC, both + in the tools and the native section. Remove all GCC, GMP, MPFR + and MPC objdirs or build once without -u. + +20170402: + a new version of dhcpcd has been imported, which does not support + update builds from the previous version. Remove your + external/bsd/dhcpcd object dir or build once without -u. + +20170211: + a new terminfo database has been imported. + The structure of it has changed slightly from prior versions and + an updated tic tool is required. + If you build.sh, don't use -u + +20170207: + various arch dependent libc/exect.S files were removed + Either remove the obj directories (lib/libc and compat/amd64/i386/lib + if it exists) or do a clean build. + (This is a bug in the make system, it should be corrected without + human intervention, but isn't.) + +20170104: + xinput build options have changed. + Remove the obj directory (external/mit/xorg/bin/xinput) + if you build.sh -u + +20170103: + a new version of flex has been imported. + Remove the file from obj (external/bsd/flex) + if you build.sh -u + +20161014: + a new version of OpenSSL has been imported. + Remove the files from obj (crypto/external/bsd/openssl) + if you build.sh -u + +20161009: + a new version of dhcpcd has been imported with slightly changed + build infrastructure. When doing a build.sh -u this requires + pruning the external/bsd/dhcpcd objdir. + +20160914: + i386, amd64, shark, ofppc and macppc have joined shark and x68k + ports in using xorg-server 1.18. This requires a clean destdir + and a clean objdir. + +20160527: + i386 needs a full cleandir or objdir deletion because PIE has + been enabled. (see the amd64 entry two down for further info) + +20160418: + libedit needs manual removal of all autogenerated files since + some of them are not autogenerated anymore. Remember that there + might be two copies of libedit if your platform builds "compat". + +20160410: + amd64 needs full "make cleandir" or deletion of objdir now that + PIE has been enabled for amd64. PIE, or position-independent + executables, means all code, including executables and not just + shared libraries, is position-independent and hence able to be + relocated by ASLR, address space layout randomization. + + The change was made in Makefile variables for compiler and + linker flags, for which make(1) does not record dependencies, + hence it is unable to detect that all .o files need rebuilding. + + Partial rebuilds with some modified source files will likely + fail when linking executables, since the linker refuses to mix + position-independent code with position-dependent code in + position-independent executables: + + .../x86_64--netbsd/bin/ld: foo.o: relocation R_X86_64_32 against `...' cannot be used when making a shared object; recompile with -fPIC + +20160401: + Ports switching to GCC 5.3 will need a full "make cleandir". + Some people have found that cleandir is not sufficient, so if + in doubt delete the entire object directory tree for gcc. + +20160306: + NetBSD's regexp implementation is now part of libnbcompat. + Nblex, nbm4, nbpax, nbsed use it and they may crash on + non-NetBSD hosts if linked with stale object files produced + while they included a non-NetBSD regex.h. + In any case, you should "make cleandir" at least these tools + before updating your toolchain. + +20160125: + Dtrace has been enabled by default on some architectures. + When doing an update build, make sure to clean the etc/mtree + object directory before starting the build - otherwise the + needed directories in destdir will not be created. + An easy way to do this is: + cd src/etc/mtree && $TOOLDIR/bin/nbmake-$arch cleandir + +20150818: + New acpica requires "make cleandir" in src/external/bsd/acpica + again. + +20150413: + New acpica requires "make cleandir" and reinstalling yacc + in /usr/src/external/bsd/byacc and /usr/src/tools/yacc and + also "make cleandir" in /usr/src/external/bsd/acpica. + +20150404: + Lint changes require a full rebuild of the tool, so make + sure to build without -u option to build.sh, or manually + do a make cleandir in src/tools/lint1 and + src/usr.bin/xlint. + +20150310: + Improvements to openssl for arm mean that update builds of + the openssl libraries will fail. A make cleandir in + external/bsd/openssl/lib is needed + +20141026: + A mishap during the import of pppd may cause your corruption + in your cvs directory if you happened to do a cvs update + during a short period of time. To fix, just remove the + directory src/external/bsd/ppp/dist/pppd completely + and let cvs restore it on next update. + +20140721: + The src/external/mit/lua/src directory was accidentally created + and then deleted in the CVS repository. If you get errors like + + cvs [update aborted]: cannot open directory + /cvsroot/src/external/mit/lua/src: No such file or directory + + then delete your local copy of that directory tree and try again. + +20140530: + ARM eABI switched to DWARF based exception handling. This requires + rebuilding all C++ code. It is strongly advised to do a clean build. + +20140131: + The new compiler_rt/libc integration moved a few things. It is strongly + advised to do a clean build. At least lib/libc, the compat version(s) + of libc, libkern, rump and the kernels need to be cleaned. + +20131227: + 1. The new ntpd runs in a restricted mode to prevent amplification + attacks. If you need ntpdc to work you need to explicitly enable + mode7 in your config file. Make sure you put the necessary restrict + statements to avoid being exposed. + 2. strncat has moved from lib/libc to common/lib/libc; you might need + to make clean in libc + +20131129: + The GMP sources were updated, and builds will likely fail without + cleaning their build trees for both tools and in-tree, like below. + +20131128: + The MPC and MPFR sources were updated, and builds may require their + tools and in-tree directories cleaned for successful updates. + +20130605: + The kernel option FAST_IPSEC no longer exists, it's been renamed + to IPSEC (and the older IPSEC version removed). + +20130605: + Previous freetype installations eroneously installed private + header files. If you are building against a non-empty $DESTDIR, + please remove ${DESTDIR}//usr/X11R7/include/freetype2/freetype/. + +20130531: + The xdm update may cause build failure due to xdm.man being + in the obj tree. Make sure to ensure any like this: + nbmake: nbmake: don't know how to make xdm.man. Stop + is fixed by deleting the xdm.man in the obj tree. + +20130530: + Updates of many xsrc packages will leave old .pc files around. + Best to clean out the xsrc objdir entirely before rebuilds. + +20130301: + The removal of netiso requires manual removal of /usr/include/netiso + prior to the build and make cleandir in /usr/src/usr.bin/{ktruss,kdump} + +20120726: + The update of OpenSSL requires cleaning both the OpenSSL build + directory and DESTDIR. *Even non-update builds require cleaning + DESTDIR.* Builds done without taking these steps may fail, or in + some cases may succeed and install broken OpenSSL libraries that + cause third-party software to link incorrectly and/or crash. + +20120507: + The database schema for makemandb was changed. You will + need to update the database using 'makemandb -f' or wait + for the next weekly run to fix it. + +20120319: + sys/conf/Makefile.kern.inc has been modified to adjust the + size of db_symtab automatically. You need to update dbsym in + your $TOOLDIR to build kernels with options SYMTAB_SPACE. If + you don't want this behavior, add AUTO_SYMTAB_SPACE=no to your + mk.conf. + +20120216: + Default for MKCATPAGES changed to NO. Update builds will fail + unless DESTDIR is cleaned manually. If you built between 20120207 + and 20120216, daily and weekly could have created an unreadable + /var/db/man.db index for apropos. Running makemandb -f or + the next run of weekly will fix it. + +20111227: + If you built between 20111225 and 20111227 you need to remove + /usr/lib/libpam.so.4* and /usr/lib/security/*.so.4, since the + bump has been reverted. + +20111125: + The "rnd" pseudodevice has been added to sys/conf/std, which + means it should no longer be explicitly listed in kernel + configuration files. The line "pseudo-device rnd" should be + removed from any custom kernel config files users may have. + +20111119: + A problem with the datastructures used by the rndctl(8) + utility (pointers in datastructures in an array, making 32->64 + bit compatibility very painful) has been fixed in a + non-backwards-compatible way. If you replace your kernel, + replace your rndctl executable too. + +20111001: + the prop_*_send_syscall() functions from proplib(3) have been + changed and their new version is not backward compatible with the old + one. So ensure that all consumers of these functions (currently: + quota2 code and its tests) are updated together with the new lib. + +20110817: + sparc has been changed to use GCC 4.5.3, so any objdir or + DESTDIR for them should be deleted before updating. + +20110806: + i386 and amd64 have been changed to use GCC 4.5.3, so any + objdir or DESTDIR for them should be deleted before updating. + +20110805: + The update to GCC 4.5.3 requires a non-trivial portion of + the tree to be cleaned. Best to delete both objdir and + DESTDIR before running this update. So far, only the + sparc64, mips and powerpc platforms have changed. + +20110803: + The layout of external/public-domain/xz has changed. To do an + update build you will have to remove the contents of the OBJDIR + for external/public-domain/xz/bin by hand as the xz entry there + is now a directory. + +20110410: + The configuration of src/tools/gcc has changed. To do an + update build you have to clean both tools/binutils and + tools/gcc by hand. + +20110328: + Building the Xorg binary was moved into a subdirectory to fix + ordering issues with "make all". It may be necessary to remove + the OBJDIR for external/mit/xorg/server/xorg-server/hw/xfree86 + if your update build fails, as the "Xorg" entry there is now a + directory. + +20110121: + Assembler files no longer use -traditional-cpp. This can break + the build of individual parts of the tree. This is handled + correctly by build.sh. Manual builds have to update /usr/share/mk + and re-run config(1) for any kernel configurations as needed. + +20101217: + The tcpdump(8) program was changed to drop privileges and chroot(2) + by default. It may be necessary to manually update passwd(5) and + group(5) in order to make the program work with existing setups. + +20101125: + The latest changes to setenv(3) dissallow setting environment + variables with names that contain '='. Revision 1.18 of env.c + assumed that this was allowed. Installing a new libc with an + old copy of /usr/bin/env causes env x=1 printenv | grep x= to + break which affects the autoconf tests for dependency finding, + so building gcc will end up printing: + checking dependency style of gcc... none + configure: error: no usable dependency style found + Fix it by rebuilding and re-installing env. + +20101119: + Recent Xorg updates in xsrc/external/mit/ may cause various build + or run-time problems. Delete your entire DESTDIR and OBJDIR if you + have any build problems with xsrc, or problems with mismatched + versions between xorg-server and drivers. + +20100604: + The update of ATF to 0.9 causes old tests written in shell to fail + unless they are rebuilt. If you are building with MKUPDATE=yes, + you need to clean the src/external/bsd/atf/tests/ and the src/tests/ + trees by hand. + +20100522: + Recent Xorg updates in xsrc/external/mit/ will cause various build + problems. Delete your entire DESTDIR and OBJDIR if you have any + build problems with xsrc. + +20100522: + private section of was splitted, and now mklocale(1) + include ctype_local.h, so you have to make cleandir in tools/mklocale. + +20100520: + The location of the xkb compiled descriptions has changed. Please + remove usr/X11R7/lib/X11/xkb/compiled from your $DESTDIR. + +20100222: + The shared objects file extension has been changed from .so to + .pico, in order to avoid conflicts with shared libraries names + libXX.so. All now stale regular .so files can be removed from + your object directories. + +20100204: + The termcap database has been removed from the sources, + but has not been marked obsolete so it is not removed + from the system when upgrading. + As such, you will need to remove them from your object + and destination directories. + +20091101: + After updating, it may be necessary to make the 'cleandir' + target in src/tools/yacc/ and in src/usr.bin/yacc/ before a + 'build.sh -u tools' or 'build.sh -u distribution'. Ditto + src/tools/lex/ and src/usr.bin/lex/. + +20091001: + On amd64 you must rebuild tools (to get the new binutils) + before building a kernel, or the build fails on cpufunc.S. + +20091001: + An error will create a ./usr/X11R7/lib/X11/xkb/compiled/xkb + symlink, failing the build. Delete the link, and the subdir + it is in, and retry your build. + +20090718: + libc build changed so that strchr() provides the extra entry + point for index(). Update build of libc.a (and libc_pic.a) + may fail because the archive contains the unwanted index.o. + (Similarly for strrchr() and rindex().) + +20090709: + Native Xorg was upgraded again. Builds will probably fail again + without a clean objdir, at least for src/external/mit/xorg. + +20090616: + Native Xorg was upgraded. Builds will need a clean objdir for + src/external/mit/xorg. Upgrading a system from sets will not + work properly yet as the /usr/X11R7/lib/X11/xkb/symbols/pc + subdirectory has been changed into a file, and this needs to + be manually "rm -r"'ed before installing xbase.tgz. + +20090501: + Several new functions were added to string.h/libc, and this + can cause autoconf problems during the tool build for people + who fail to clean out their tools objects properly. If you + note messages about stpcpy, stpncpy or strnlen accompanying a + failure during the tool build, clean out all your tools + objects and start again. + +20090325: + The i386 port was switched to i486 default toolchain. This requires + cleaning your src/tools directory and $TOOLDIR and rebuilding them. + +20090126: + The __posix_fadvise50 system call changed assembly stub type. You + need to 'rm -f __posix_fadvise50.*' in the libc build directory + to avoid using the old assembly stub. + +20090202: + pkg_install now depends on the pkgdb cache for automatic conflict + detection. It is recommented to rebuild the cache with + ``pkg_admin rebuild''. + +20090110: + time_t and dev_t have been bumped to 64 bit quantities. To upgrade: + 1. Make sure your kernel has COMPAT_50 in it. Build and install. + This is needed even in the MODULAR kernel because there is + conditionally compiled code in rtsock.c. + 2. make sure build.sh completes and the binaries in a chroot work + before installing. + 3. If you don't use build.sh and you build directly to root, and + your build breaks in the middle, don't despair. Make sure headers + are installed properly, and start building libraries first libc + and libutil, install them and then continue building all the + libraries in src/lib and src/gnu/lib and install them. Once + the new libraries are installed, you can restart the build. + 4. If you compile packages and you notice link time warnings, + rebuild the required packages to update their shared libraries. + Any package you rebuild will require rebuilding all the packages + that depend on it. + 5. Next time you run pwd_mkdb with the new binary, the file + will be upgraded and it will not be backwards compatible. + 6. The utmpx/wtmpx files (/var/run/utmpx and /var/log/wtmpx, see + lastlogx(5)) have been versioned, and there is a heuristic + for utmp. You are better off removing the old files after + upgrading. The automated clearing of /var/run during + boot, and the automated rotating of files in /var/log by + newsyslog(8), may mean that you do not have to remove the + files manually. + 7. The optional accounting file (/var/account/acct, see + accton(8)) has not been versioned, and will need to be + removed. The automatic rotation of the accounting file by + /etc/daily limits the bad consequences of failure to remove + the file. + 8. Application software that writes time_t to binary files on + disk will break or need attention. Most notably: if you are + using PostgreSQL < 8.4, you need to dump your databases, + rebuild PostgreSQL with the new time_t, then restore. + +20081219: + config(1) has been updated, and one of the files it creates - + swapnetbsd.o - has changed format. You need to rebuild config + (done automatically by build.sh) and then you need to rerun + config on all kernel configuration files before rebuilding those + kernels. + +20081205: + If you build with MKX11=no, you should remove /etc/rc.d/xdm and + /etc/rc.d/xfs from DESTDIR because those files were moved to the xetc + set and will appear as extra files for MKX11=no update builds. + +20081122: + On i386, various kernel options(4) in GENERIC including + file systems have been disabled and moved into kernel modules. + Before trying a new GENERIC kernel, you have to prepare the + following files as well as a new GENERIC kernel: + + - build and install kernel modules from src/sys/modules + + - install the latest bootloader, which will load a module + for the file system from which the kernel is loaded automatically + + If you have to load your kernel from a file system which is not of + the same type as the root file system, you have to load the necessary + file system module manually on the boot prompt or in the boot.cfg file. + +20080827: + If you built and installed a libc from sources between + 2008/08/20 and 2008/08/26 you got a broken strtouq(3) + which results in false errors reported by lint(1). + Since this breaks the libc build itself, manual help is + needed -- lint must be disabled temporarily, e.g.: + $ (cd lib/libc && make MKLINT=no dependall install) + +20080813: + MKDEBUG build was broken because the .depend files did not know + about .go files. You need to remove all .depend files and rebuild. + +20080802: + A regression in binary compatibility for pthread_mutex_t has + been fixed. Unfortunately, the price is breaking compatibility + for -current. + + Threaded programs (using libpthread) and C++ programs (using + libstdc++) compiled after 20070907 and before 20080802 need to + be recompiled. + + One way to find affected pkgsrc packages: + + $ grep REQUIRES=/usr/lib/libpthread /var/db/pkg/*/+BUILD_INFO + $ grep REQUIRES=/usr/lib/libstdc++ /var/db/pkg/*/+BUILD_INFO + +20080731: + WAPBL (metadata journaling support) has been added, but at this + time isn't backwards compatible with pre-WAPBL aware kernels + and userland (fsck_ffs in particular). Please make sure you + don't use a journaled filesystem with an older kernel/userland, + especially an uncleanly mounted journaled filesystem. WAPBL + also requires the super block to be in the UFS2 format. You + can use fsck_ffs -c 4 to update the superblock format. + +20080721: + Assembler warnings are now fatal if $WARNS>0 and $NOGCCERROR + isn't defined. + +20080531: + The ioctl number of DRVSUSPENDDEV command on /dev/drvctl changed + from 125 (conflicted with DRVCTLCOMMAND) to 129. The drvctl(8) + utility needs to be rebuilt and reinstalled as older binaries + won't work correctly. The following sequence of commands: + + $ (cd sys/sys/ && nbmake-$arch includes) + $ (cd sbin/drvctl/ && nbmake-$arch clean) + $ (cd sbin/drvctl/ && nbmake-$arch all) + + leaves new drvctl utility in sbin/drvctl build directory. + +20080503: + The variable MKPRIVATELIB was renamed to LIBISPRIVATE. + +20080521: + For a while, unprivileged UPDATE builds would fail to + succeed at the checkflist stage, complaining that + ${DESTDIR}/stand// did not exist. A fix for this + problem was committed to share/mk/bsd.kmodule.mk, revision 1.9. + If you already hit this problem, update the .mk file, + remove ${DESTDIR}/stand/, and re-run the build. + +20080303: + Linker warnings are now fatal if $WARNS>0. + +20080126: + The posix_fadvise system call has been changed from an assembly + stub, to a c file that calls an assembly stub. You need to + 'rm -f posix_fadvise.* .depend' in the libc build directory to + avoid using the old assembly stub. + +20071209: + The acpiec(4) driver has been split into two attachments. If you + get ACPI errors before the attachment, please update your kernel + configuration file appropriately or see GENERIC for more details. + +20071115: + The it(4) driver has been renamed to itesio(4) and the old port + argument specified in the kernel configuration file is not valid + anymore. The itesio(4) driver now uses the Super I/O address port + rather than the EC address port. Please update your kernel + configuration file appropriately or see GENERIC for more details. + +20071028: + The pccons(4) driver has been removed from the NetBSD/shark port. + You need to update any custom kernel configuration file you have + to remove any references to pccons (which includes removing the + now useless XSERVER option) and replace them with the correct + entries for the wscons driver. See the GENERIC configuration file + for more details. + +20070913: + A latent bug in dhclient/dhcpd that caused it to be unable to + enumerate interfaces was fixed. The bug began to cause + problems after 20070911 when the kernel's SIOCGIFCONF + implementation was repaired. From 20070529 to 20070911 racoon + could not enumerate interfaces. (These are noted because + normal kernel/userspace version matching hygiene is not + sufficient to avoid this problem.) Ensure that both kernel + and userland are from after 20070913. + +20070703: + nbinstall has been renamed ${MACHINE_GNU_ARCH}-install. It + calls the target-specific strip program, and the logic to pass + down STRIP from make has been removed. This requires a + re-installation of tools. + +20070422: + The way OS emulations lookup filenames inside the emulation root + has been changed. Rather than modify the pathname (and copy back + to userspace) namei() and lookup() directly check the emulation + root. One side effect is that absolute symlinks inside the emulated + root file system will be relative to that file system - unless they + start /../ this is useful when the emulated root is a real install + that has such links. + This might affect symlinks that have been added to reference outside + the emulated root. + +20070412: + The pckbc driver on sgimips IP32 has been removed. Use macekbc + instead. See the GENERIC32_IP3x kernel configuration for an + example. + +20070319: + src/lib/libc/Makefile revision 1.129 broke libc and ld.elf_so + on many platforms due to incorrect flags settings. If you + updated and built after about 20070315, do "nbmake-$arch + cleandir" in src/lib/libc and src/libexec/ld.elf_so to force a + rebuild of object files that might have been built + incorrectly, and ensure that you have at least + src/lib/libc/Makefile 1.130. + +20070210: + src/sys/sys/{sa.h,savar.h} were removed. + find ${OBJDIR} \( -name .depend -o -name '*.d' \) -print \ + | xargs egrep -l '/sa.h|/savar.h' | xargs rm + will allow dependencies on those files to get get rebuilt + +20070209: + The threading model was changed when the newlock2 branch + was merged to NetBSD-current. If you boot with a new + kernel (version 4.99.10), then you also need a new pthread + library (/usr/lib/libpthread.so.0.7). If you boot with + an old kernel, then you need the old pthread library + (/usr/lib/libpthread.so.0.6). Provided you keep the kernel and + the pthread library in sync, old threaded applications should + continue to work with an old or new kernel. Note that named(8) + is the only threaded application in the base system. + +20061214: + Following the move of string_to_flags() and flags_to_string() + from the bin/ls/ sources to libutil, users doing UPDATE builds + will need to do a "make cleandir" in + tools/mtree/, tools/makefs/, tools/binstall/, tools/pax/, + bin/pax/, bin/ls/, usr.sbin/mtree/, usr.sbin/makefs/, + usr.bin/xinstall/, libexec/ftpd/, rescue/, as well + as the installation images in distrib/ + in order to excise stale references to the old stat_flags.h header + file in the ls sources -- stat_flags.h has been removed. + +20061108: + The configure script used in the src/tools/gcc compiler has been + changed to indicate that our libc has ssp support built-in and + does not depend on -lssp and -lssp-nonshared. You'll need to + make clean in src/tools/gcc first to rebuild the compiler. + +20061009: + The sysctl variables net.inet{,6}.tcp{,6}.newreno are no longer + available. Use net.inet{,6}.tcp{,6}.congctl.selected instead. + +20060814: + The vt, vidcconsole, kbd, and rpckbd drivers on acorn32 have been + withdrawn. Use vidcvideo and pckbd instead. See the GENERIC + kernel configuration for an example. X servers from the last + few years should cope. + +20060703: + MPACPI is no more. We always configure PCI interrupts using ACPI + if we have an ACPI kernel. The option MPACPI_SCANPCI has been renamed + to ACPI_SCANPCI. Thanks to work from fvdl. + +20060627: + socket(2) has changed, and its system call has been versioned. + For userlands with the old version of socket(2), make sure that + your kernel has 'options COMPAT_30' set, or else 'bad system call' + errors will result. + +Hints for a more successful build: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Use build.sh, but do not use its "expert mode": + This will automatically build the tools in the + correct order, and it will keep the tools and the + new build products from interfering with the running + system. This will allow you to ignore most of the + other advice in this file. + Build a new kernel first: + This makes sure that any new system calls or features + expected by the new userland will be present. This + helps to avoid critical errors when upgrading. + Use object directories: + This helps to keep stale object + files from polluting the build if a Makefile "forgets" + about one. It also makes it easier to clean up after + a build. It's also necessary if you want to use the + same source tree for multiple machines. + To use object directories with build.sh: + a) invoke build.sh with the "-M" or "-O" options. + To use object directories without using build.sh: + a) cd /usr/src ; make cleandir + b) Add "OBJMACHINE=yes" to /etc/mk.conf + c) Add "MKOBJDIRS=yes" to /etc/mk.conf + d) cd /usr/src ; make build + Note that running "make obj" in a directory will create + in obj.$MACHINE directory. + Build to a DESTDIR: + This helps to keep old installed files (especially libraries) + from interfering with the new build. + To build to a DESTDIR with build.sh, use the "-D" option. + To build to a DESTDIR without using build.sh, set the DESTDIR + environment variable before running make build. It should be + set to the pathname of an initially empty directory. + Problems: if you do not use build.sh, you might need to + update critical utilities without using DESTDIR since + nothing is executed from what is installed in DESTDIR. + (See critical utils, below.) + Build often: + This keeps critical utilities current enough to not choke + on any other part of the source tree that depends on up to + date functionality. If you use build.sh, you should not have + this problem. + +What to do if things don't work: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +When things don't work there are usually a few things that commonly +should be done. + 1) make includes + This should be done automatically by make build. + 2) cd share/mk && make install + Again, automatically done by make build. + +Failsafe rebuild of a small part of the tree: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +To make sure you rebuild something correctly you want to do +something like the following: + 1) Make sure the includes and .mk files are up to date. + 2) Make sure any program used to build the particular + utility is up to date. (yacc, lex, etc...) + 3) cd ...path/to/util... + make cleandir + rm ...all obj directories... + make cleandir # yes, again + make obj + make depend && make + +Failsafe rebuild of the entire tree: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +If you really want to make sure the source tree is clean and +ready for a build try the following. Note that sourcing /etc/mk.conf +(a make(1) Makefile) in this manner is not right, and will not work +for anyone who uses any make(1) features in /etc/mk.conf. + +---cut here--- +#!/bin/sh +. /etc/mk.conf + +if [ -z $NETBSDSRCDIR ] ; then + NETBSDSRCDIR=/usr/src +fi +if [ \! -d $NETBSDSRCDIR ] ; then + echo Unable to find sources + exit 1 +fi +find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; + +if [ -z $BSDOBJDIR ] ; then + BSDOBJDIR=/usr/obj +fi +if [ -d $BSDOBJDIR ] ; then + rm -rf $BSDOBJDIR +fi + +cd $NETBSDSRCDIR && make cleandir + +---cut here--- + +Critical utilities: +^^^^^^^^^^^^^^^^^^^ + usr.bin/compile_et + usr.bin/make + usr.bin/yacc + usr.bin/lex + usr.bin/xlint + usr.bin/config + +Other problems and possible solutions: +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Symptom:Complaints involving a Makefile. +Fix: Rebuild usr.bin/make: + cd usr.bin/make && make && make install + Or, a failsafe method if that doesn't work: + cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin + +Fix: Make sure .mk files are up to date. + cd share/mk && make install + +Symptom:Kernel `config' fails to configure any kernel, including GENERIC. +Fix: Rebuild usr.bin/config + +Symptom: +Fix: Rebuild usr.bin/yacc + +Symptom: +Fix: Rebuild usr.bin/lex + +Symptom: +Fix: rm /usr/lib/libbfd.a + +Symptom:Obsolete intermediate files are used during compilation +Fix: Try the following sequence of commands in the directory in question. + make cleandir; rm `make print-objdir`; make cleandir; make obj + (If you built the tree without "make obj" in the past, obsolete files + may remain. The command tries to clean everything up) + +Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible + pointer type +Fix: Rebuild and install usr.bin/menuc + +Symptom:mklocale not found during build in share/locale/ctype +Fix: Build and install usr.bin/mklocale + +Symptom:undefined reference to `__assert13' or `__unsetenv13' +Fix: Rebuild and install lib/libc + +Symptom:usr.bin/config fails to build. +Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. + +Symptom:undefined reference to `getprogname' or `setprogname' +Fix: Rebuild and install lib/libc + +Symptom:lint does not understand the '-X' option +Fix: May need to build & install libs with NOLINT=1 before rebuilding lint + +Symptom:Update build fails in src/tools/gcc complaining that a variable + (e.g. CPPFLAGS) has changed since the previous run. +Fix: Run "nbmake-${ARCH} clean" in src/tools/gcc or do a clean build. + +Symptom:cvs [update aborted]: cannot open directory /cvsroot/src/...: + No such file or directory. +Cause: If a directory is created by mistake, then it is sometimes + deleted from the CVS repository using administrative commands + that bypass the normal cvs access controls. If your cvs working tree + contains references to a directory that has been deleted on the + server in this way, then "cvs update" reports this error. +Fix: Recursively delete the affected directory from your working tree + and try the update again. diff --git a/build.sh b/build.sh new file mode 100755 index 000000000000..44261664f8a1 --- /dev/null +++ b/build.sh @@ -0,0 +1,2399 @@ +#! /usr/bin/env sh +# $NetBSD: build.sh,v 1.327 2018/05/02 07:34:44 pgoyette Exp $ +# +# Copyright (c) 2001-2011 The NetBSD Foundation, Inc. +# All rights reserved. +# +# This code is derived from software contributed to The NetBSD Foundation +# by Todd Vierling and Luke Mewburn. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS +# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED +# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS +# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +# POSSIBILITY OF SUCH DAMAGE. +# +# +# Top level build wrapper, to build or cross-build NetBSD. +# + +# +# {{{ Begin shell feature tests. +# +# We try to determine whether or not this script is being run under +# a shell that supports the features that we use. If not, we try to +# re-exec the script under another shell. If we can't find another +# suitable shell, then we print a message and exit. +# + +errmsg='' # error message, if not empty +shelltest=false # if true, exit after testing the shell +re_exec_allowed=true # if true, we may exec under another shell + +# Parse special command line options in $1. These special options are +# for internal use only, are not documented, and are not valid anywhere +# other than $1. +case "$1" in +"--shelltest") + shelltest=true + re_exec_allowed=false + shift + ;; +"--no-re-exec") + re_exec_allowed=false + shift + ;; +esac + +# Solaris /bin/sh, and other SVR4 shells, do not support "!". +# This is the first feature that we test, because subsequent +# tests use "!". +# +if test -z "$errmsg"; then + if ( eval '! false' ) >/dev/null 2>&1 ; then + : + else + errmsg='Shell does not support "!".' + fi +fi + +# Does the shell support functions? +# +if test -z "$errmsg"; then + if ! ( + eval 'somefunction() { : ; }' + ) >/dev/null 2>&1 + then + errmsg='Shell does not support functions.' + fi +fi + +# Does the shell support the "local" keyword for variables in functions? +# +# Local variables are not required by SUSv3, but some scripts run during +# the NetBSD build use them. +# +# ksh93 fails this test; it uses an incompatible syntax involving the +# keywords 'function' and 'typeset'. +# +if test -z "$errmsg"; then + if ! ( + eval 'f() { local v=2; }; v=1; f && test x"$v" = x"1"' + ) >/dev/null 2>&1 + then + errmsg='Shell does not support the "local" keyword in functions.' + fi +fi + +# Does the shell support ${var%suffix}, ${var#prefix}, and their variants? +# +# We don't bother testing for ${var+value}, ${var-value}, or their variants, +# since shells without those are sure to fail other tests too. +# +if test -z "$errmsg"; then + if ! ( + eval 'var=a/b/c ; + test x"${var#*/};${var##*/};${var%/*};${var%%/*}" = \ + x"b/c;c;a/b;a" ;' + ) >/dev/null 2>&1 + then + errmsg='Shell does not support "${var%suffix}" or "${var#prefix}".' + fi +fi + +# Does the shell support IFS? +# +# zsh in normal mode (as opposed to "emulate sh" mode) fails this test. +# +if test -z "$errmsg"; then + if ! ( + eval 'IFS=: ; v=":a b::c" ; set -- $v ; IFS=+ ; + test x"$#;$1,$2,$3,$4;$*" = x"4;,a b,,c;+a b++c"' + ) >/dev/null 2>&1 + then + errmsg='Shell does not support IFS word splitting.' + fi +fi + +# Does the shell support ${1+"$@"}? +# +# Some versions of zsh fail this test, even in "emulate sh" mode. +# +if test -z "$errmsg"; then + if ! ( + eval 'set -- "a a a" "b b b"; set -- ${1+"$@"}; + test x"$#;$1;$2" = x"2;a a a;b b b";' + ) >/dev/null 2>&1 + then + errmsg='Shell does not support ${1+"$@"}.' + fi +fi + +# Does the shell support $(...) command substitution? +# +if test -z "$errmsg"; then + if ! ( + eval 'var=$(echo abc); test x"$var" = x"abc"' + ) >/dev/null 2>&1 + then + errmsg='Shell does not support "$(...)" command substitution.' + fi +fi + +# Does the shell support $(...) command substitution with +# unbalanced parentheses? +# +# Some shells known to fail this test are: NetBSD /bin/ksh (as of 2009-12), +# bash-3.1, pdksh-5.2.14, zsh-4.2.7 in "emulate sh" mode. +# +if test -z "$errmsg"; then + if ! ( + eval 'var=$(case x in x) echo abc;; esac); test x"$var" = x"abc"' + ) >/dev/null 2>&1 + then + # XXX: This test is ignored because so many shells fail it; instead, + # the NetBSD build avoids using the problematic construct. + : ignore 'Shell does not support "$(...)" with unbalanced ")".' + fi +fi + +# Does the shell support getopts or getopt? +# +if test -z "$errmsg"; then + if ! ( + eval 'type getopts || type getopt' + ) >/dev/null 2>&1 + then + errmsg='Shell does not support getopts or getopt.' + fi +fi + +# +# If shelltest is true, exit now, reporting whether or not the shell is good. +# +if $shelltest; then + if test -n "$errmsg"; then + echo >&2 "$0: $errmsg" + exit 1 + else + exit 0 + fi +fi + +# +# If the shell was bad, try to exec a better shell, or report an error. +# +# Loops are broken by passing an extra "--no-re-exec" flag to the new +# instance of this script. +# +if test -n "$errmsg"; then + if $re_exec_allowed; then + for othershell in \ + "${HOST_SH}" /usr/xpg4/bin/sh ksh ksh88 mksh pdksh dash bash + # NOTE: some shells known not to work are: + # any shell using csh syntax; + # Solaris /bin/sh (missing many modern features); + # ksh93 (incompatible syntax for local variables); + # zsh (many differences, unless run in compatibility mode). + do + test -n "$othershell" || continue + if eval 'type "$othershell"' >/dev/null 2>&1 \ + && "$othershell" "$0" --shelltest >/dev/null 2>&1 + then + cat <&2 < $@" | tee -a "${results}" +} + +statusmsg2() +{ + local msg + + msg="${1}" + shift + case "${msg}" in + ????????????????*) ;; + ??????????*) msg="${msg} ";; + ?????*) msg="${msg} ";; + *) msg="${msg} ";; + esac + case "${msg}" in + ?????????????????????*) ;; + ????????????????????) msg="${msg} ";; + ???????????????????) msg="${msg} ";; + ??????????????????) msg="${msg} ";; + ?????????????????) msg="${msg} ";; + ????????????????) msg="${msg} ";; + esac + statusmsg "${msg}$*" +} + +warning() +{ + statusmsg "Warning: $@" +} + +# Find a program in the PATH, and print the result. If not found, +# print a default. If $2 is defined (even if it is an empty string), +# then that is the default; otherwise, $1 is used as the default. +find_in_PATH() +{ + local prog="$1" + local result="${2-"$1"}" + local oldIFS="${IFS}" + local dir + IFS=":" + for dir in ${PATH}; do + if [ -x "${dir}/${prog}" ]; then + result="${dir}/${prog}" + break + fi + done + IFS="${oldIFS}" + echo "${result}" +} + +# Try to find a working POSIX shell, and set HOST_SH to refer to it. +# Assumes that uname_s, uname_m, and PWD have been set. +set_HOST_SH() +{ + # Even if ${HOST_SH} is already defined, we still do the + # sanity checks at the end. + + # Solaris has /usr/xpg4/bin/sh. + # + [ -z "${HOST_SH}" ] && [ x"${uname_s}" = x"SunOS" ] && \ + [ -x /usr/xpg4/bin/sh ] && HOST_SH="/usr/xpg4/bin/sh" + + # Try to get the name of the shell that's running this script, + # by parsing the output from "ps". We assume that, if the host + # system's ps command supports -o comm at all, it will do so + # in the usual way: a one-line header followed by a one-line + # result, possibly including trailing white space. And if the + # host system's ps command doesn't support -o comm, we assume + # that we'll get an error message on stderr and nothing on + # stdout. (We don't try to use ps -o 'comm=' to suppress the + # header line, because that is less widely supported.) + # + # If we get the wrong result here, the user can override it by + # specifying HOST_SH in the environment. + # + [ -z "${HOST_SH}" ] && HOST_SH="$( + (ps -p $$ -o comm | sed -ne "2s/[ ${tab}]*\$//p") 2>/dev/null )" + + # If nothing above worked, use "sh". We will later find the + # first directory in the PATH that has a "sh" program. + # + [ -z "${HOST_SH}" ] && HOST_SH="sh" + + # If the result so far is not an absolute path, try to prepend + # PWD or search the PATH. + # + case "${HOST_SH}" in + /*) : + ;; + */*) HOST_SH="${PWD}/${HOST_SH}" + ;; + *) HOST_SH="$(find_in_PATH "${HOST_SH}")" + ;; + esac + + # If we don't have an absolute path by now, bomb. + # + case "${HOST_SH}" in + /*) : + ;; + *) bomb "HOST_SH=\"${HOST_SH}\" is not an absolute path." + ;; + esac + + # If HOST_SH is not executable, bomb. + # + [ -x "${HOST_SH}" ] || + bomb "HOST_SH=\"${HOST_SH}\" is not executable." + + # If HOST_SH fails tests, bomb. + # ("$0" may be a path that is no longer valid, because we have + # performed "cd $(dirname $0)", so don't use $0 here.) + # + "${HOST_SH}" build.sh --shelltest || + bomb "HOST_SH=\"${HOST_SH}\" failed functionality tests." +} + +# initdefaults -- +# Set defaults before parsing command line options. +# +initdefaults() +{ + makeenv= + makewrapper= + makewrappermachine= + runcmd= + operations= + removedirs= + + [ -d usr.bin/make ] || cd "$(dirname $0)" + [ -d usr.bin/make ] || + bomb "usr.bin/make not found; build.sh must be run from the top \ +level of source directory" + [ -f share/mk/bsd.own.mk ] || + bomb "src/share/mk is missing; please re-fetch the source tree" + + # Set various environment variables to known defaults, + # to minimize (cross-)build problems observed "in the field". + # + # LC_ALL=C must be set before we try to parse the output from + # any command. Other variables are set (or unset) here, before + # we parse command line arguments. + # + # These variables can be overridden via "-V var=value" if + # you know what you are doing. + # + unsetmakeenv INFODIR + unsetmakeenv LESSCHARSET + unsetmakeenv MAKEFLAGS + unsetmakeenv TERMINFO + setmakeenv LC_ALL C + + # Find information about the build platform. This should be + # kept in sync with _HOST_OSNAME, _HOST_OSREL, and _HOST_ARCH + # variables in share/mk/bsd.sys.mk. + # + # Note that "uname -p" is not part of POSIX, but we want uname_p + # to be set to the host MACHINE_ARCH, if possible. On systems + # where "uname -p" fails, prints "unknown", or prints a string + # that does not look like an identifier, fall back to using the + # output from "uname -m" instead. + # + uname_s=$(uname -s 2>/dev/null) + uname_r=$(uname -r 2>/dev/null) + uname_m=$(uname -m 2>/dev/null) + uname_p=$(uname -p 2>/dev/null || echo "unknown") + case "${uname_p}" in + ''|unknown|*[^-_A-Za-z0-9]*) uname_p="${uname_m}" ;; + esac + + id_u=$(id -u 2>/dev/null || /usr/xpg4/bin/id -u 2>/dev/null) + + # If $PWD is a valid name of the current directory, POSIX mandates + # that pwd return it by default which causes problems in the + # presence of symlinks. Unsetting PWD is simpler than changing + # every occurrence of pwd to use -P. + # + # XXX Except that doesn't work on Solaris. Or many Linuces. + # + unset PWD + TOP=$( (exec pwd -P 2>/dev/null) || (exec pwd 2>/dev/null) ) + + # The user can set HOST_SH in the environment, or we try to + # guess an appropriate value. Then we set several other + # variables from HOST_SH. + # + set_HOST_SH + setmakeenv HOST_SH "${HOST_SH}" + setmakeenv BSHELL "${HOST_SH}" + setmakeenv CONFIG_SHELL "${HOST_SH}" + + # Set defaults. + # + toolprefix=nb + + # Some systems have a small ARG_MAX. -X prevents make(1) from + # exporting variables in the environment redundantly. + # + case "${uname_s}" in + Darwin | FreeBSD | CYGWIN*) + MAKEFLAGS="-X ${MAKEFLAGS}" + ;; + esac + + # do_{operation}=true if given operation is requested. + # + do_expertmode=false + do_rebuildmake=false + do_removedirs=false + do_tools=false + do_cleandir=false + do_obj=false + do_build=false + do_distribution=false + do_release=false + do_kernel=false + do_releasekernel=false + do_kernels=false + do_modules=false + do_installmodules=false + do_install=false + do_sets=false + do_sourcesets=false + do_syspkgs=false + do_iso_image=false + do_iso_image_source=false + do_live_image=false + do_install_image=false + do_disk_image=false + do_params=false + do_rump=false + + # done_{operation}=true if given operation has been done. + # + done_rebuildmake=false + + # Create scratch directory + # + tmpdir="${TMPDIR-/tmp}/nbbuild$$" + mkdir "${tmpdir}" || bomb "Cannot mkdir: ${tmpdir}" + trap "cd /; rm -r -f \"${tmpdir}\"" 0 + results="${tmpdir}/build.sh.results" + + # Set source directories + # + setmakeenv NETBSDSRCDIR "${TOP}" + + # Make sure KERNOBJDIR is an absolute path if defined + # + case "${KERNOBJDIR}" in + ''|/*) ;; + *) KERNOBJDIR="${TOP}/${KERNOBJDIR}" + setmakeenv KERNOBJDIR "${KERNOBJDIR}" + ;; + esac + + # Find the version of NetBSD + # + DISTRIBVER="$(${HOST_SH} ${TOP}/sys/conf/osrelease.sh)" + + # Set the BUILDSEED to NetBSD-"N" + # + setmakeenv BUILDSEED "NetBSD-$(${HOST_SH} ${TOP}/sys/conf/osrelease.sh -m)" + + # Set MKARZERO to "yes" + # + setmakeenv MKARZERO "yes" + +} + +# valid_MACHINE_ARCH -- A multi-line string, listing all valid +# MACHINE/MACHINE_ARCH pairs. +# +# Each line contains a MACHINE and MACHINE_ARCH value, an optional ALIAS +# which may be used to refer to the MACHINE/MACHINE_ARCH pair, and an +# optional DEFAULT or NO_DEFAULT keyword. +# +# When a MACHINE corresponds to multiple possible values of +# MACHINE_ARCH, then this table should list all allowed combinations. +# If the MACHINE is associated with a default MACHINE_ARCH (to be +# used when the user specifies the MACHINE but fails to specify the +# MACHINE_ARCH), then one of the lines should have the "DEFAULT" +# keyword. If there is no default MACHINE_ARCH for a particular +# MACHINE, then there should be a line with the "NO_DEFAULT" keyword, +# and with a blank MACHINE_ARCH. +# +valid_MACHINE_ARCH=' +MACHINE=acorn32 MACHINE_ARCH=arm +MACHINE=algor MACHINE_ARCH=mips64el ALIAS=algor64 +MACHINE=algor MACHINE_ARCH=mipsel DEFAULT +MACHINE=alpha MACHINE_ARCH=alpha +MACHINE=amd64 MACHINE_ARCH=x86_64 +MACHINE=amiga MACHINE_ARCH=m68k +MACHINE=amigappc MACHINE_ARCH=powerpc +MACHINE=arc MACHINE_ARCH=mips64el ALIAS=arc64 +MACHINE=arc MACHINE_ARCH=mipsel DEFAULT +MACHINE=atari MACHINE_ARCH=m68k +MACHINE=bebox MACHINE_ARCH=powerpc +MACHINE=cats MACHINE_ARCH=arm ALIAS=ocats +MACHINE=cats MACHINE_ARCH=earmv4 ALIAS=ecats DEFAULT +MACHINE=cesfic MACHINE_ARCH=m68k +MACHINE=cobalt MACHINE_ARCH=mips64el ALIAS=cobalt64 +MACHINE=cobalt MACHINE_ARCH=mipsel DEFAULT +MACHINE=dreamcast MACHINE_ARCH=sh3el +MACHINE=emips MACHINE_ARCH=mipseb +MACHINE=epoc32 MACHINE_ARCH=arm +MACHINE=evbarm MACHINE_ARCH=arm ALIAS=evboarm-el +MACHINE=evbarm MACHINE_ARCH=armeb ALIAS=evboarm-eb +MACHINE=evbarm MACHINE_ARCH=earm ALIAS=evbearm-el DEFAULT +MACHINE=evbarm MACHINE_ARCH=earmeb ALIAS=evbearm-eb +MACHINE=evbarm MACHINE_ARCH=earmhf ALIAS=evbearmhf-el +MACHINE=evbarm MACHINE_ARCH=earmhfeb ALIAS=evbearmhf-eb +MACHINE=evbarm MACHINE_ARCH=earmv4 ALIAS=evbearmv4-el +MACHINE=evbarm MACHINE_ARCH=earmv4eb ALIAS=evbearmv4-eb +MACHINE=evbarm MACHINE_ARCH=earmv5 ALIAS=evbearmv5-el +MACHINE=evbarm MACHINE_ARCH=earmv5eb ALIAS=evbearmv5-eb +MACHINE=evbarm MACHINE_ARCH=earmv6 ALIAS=evbearmv6-el +MACHINE=evbarm MACHINE_ARCH=earmv6hf ALIAS=evbearmv6hf-el +MACHINE=evbarm MACHINE_ARCH=earmv6eb ALIAS=evbearmv6-eb +MACHINE=evbarm MACHINE_ARCH=earmv6hfeb ALIAS=evbearmv6hf-eb +MACHINE=evbarm MACHINE_ARCH=earmv7 ALIAS=evbearmv7-el +MACHINE=evbarm MACHINE_ARCH=earmv7eb ALIAS=evbearmv7-eb +MACHINE=evbarm MACHINE_ARCH=earmv7hf ALIAS=evbearmv7hf-el +MACHINE=evbarm MACHINE_ARCH=earmv7hfeb ALIAS=evbearmv7hf-eb +MACHINE=evbarm MACHINE_ARCH=aarch64 ALIAS=evbarm64-el DEFAULT +MACHINE=evbarm MACHINE_ARCH=aarch64eb ALIAS=evbarm64-eb +MACHINE=evbcf MACHINE_ARCH=coldfire +MACHINE=evbmips MACHINE_ARCH= NO_DEFAULT +MACHINE=evbmips MACHINE_ARCH=mips64eb ALIAS=evbmips64-eb +MACHINE=evbmips MACHINE_ARCH=mips64el ALIAS=evbmips64-el +MACHINE=evbmips MACHINE_ARCH=mipseb ALIAS=evbmips-eb +MACHINE=evbmips MACHINE_ARCH=mipsel ALIAS=evbmips-el +MACHINE=evbppc MACHINE_ARCH=powerpc DEFAULT +MACHINE=evbppc MACHINE_ARCH=powerpc64 ALIAS=evbppc64 +MACHINE=evbsh3 MACHINE_ARCH= NO_DEFAULT +MACHINE=evbsh3 MACHINE_ARCH=sh3eb ALIAS=evbsh3-eb +MACHINE=evbsh3 MACHINE_ARCH=sh3el ALIAS=evbsh3-el +MACHINE=ews4800mips MACHINE_ARCH=mipseb +MACHINE=hp300 MACHINE_ARCH=m68k +MACHINE=hppa MACHINE_ARCH=hppa +MACHINE=hpcarm MACHINE_ARCH=arm ALIAS=hpcoarm +MACHINE=hpcarm MACHINE_ARCH=earmv4 ALIAS=hpcearm DEFAULT +MACHINE=hpcmips MACHINE_ARCH=mipsel +MACHINE=hpcsh MACHINE_ARCH=sh3el +MACHINE=i386 MACHINE_ARCH=i386 +MACHINE=ia64 MACHINE_ARCH=ia64 +MACHINE=ibmnws MACHINE_ARCH=powerpc +MACHINE=iyonix MACHINE_ARCH=arm ALIAS=oiyonix +MACHINE=iyonix MACHINE_ARCH=earm ALIAS=eiyonix DEFAULT +MACHINE=landisk MACHINE_ARCH=sh3el +MACHINE=luna68k MACHINE_ARCH=m68k +MACHINE=mac68k MACHINE_ARCH=m68k +MACHINE=macppc MACHINE_ARCH=powerpc DEFAULT +MACHINE=macppc MACHINE_ARCH=powerpc64 ALIAS=macppc64 +MACHINE=mipsco MACHINE_ARCH=mipseb +MACHINE=mmeye MACHINE_ARCH=sh3eb +MACHINE=mvme68k MACHINE_ARCH=m68k +MACHINE=mvmeppc MACHINE_ARCH=powerpc +MACHINE=netwinder MACHINE_ARCH=arm ALIAS=onetwinder +MACHINE=netwinder MACHINE_ARCH=earmv4 ALIAS=enetwinder DEFAULT +MACHINE=news68k MACHINE_ARCH=m68k +MACHINE=newsmips MACHINE_ARCH=mipseb +MACHINE=next68k MACHINE_ARCH=m68k +MACHINE=ofppc MACHINE_ARCH=powerpc DEFAULT +MACHINE=ofppc MACHINE_ARCH=powerpc64 ALIAS=ofppc64 +MACHINE=or1k MACHINE_ARCH=or1k +MACHINE=playstation2 MACHINE_ARCH=mipsel +MACHINE=pmax MACHINE_ARCH=mips64el ALIAS=pmax64 +MACHINE=pmax MACHINE_ARCH=mipsel DEFAULT +MACHINE=prep MACHINE_ARCH=powerpc +MACHINE=riscv MACHINE_ARCH=riscv64 ALIAS=riscv64 DEFAULT +MACHINE=riscv MACHINE_ARCH=riscv32 ALIAS=riscv32 +MACHINE=rs6000 MACHINE_ARCH=powerpc +MACHINE=sandpoint MACHINE_ARCH=powerpc +MACHINE=sbmips MACHINE_ARCH= NO_DEFAULT +MACHINE=sbmips MACHINE_ARCH=mips64eb ALIAS=sbmips64-eb +MACHINE=sbmips MACHINE_ARCH=mips64el ALIAS=sbmips64-el +MACHINE=sbmips MACHINE_ARCH=mipseb ALIAS=sbmips-eb +MACHINE=sbmips MACHINE_ARCH=mipsel ALIAS=sbmips-el +MACHINE=sgimips MACHINE_ARCH=mips64eb ALIAS=sgimips64 +MACHINE=sgimips MACHINE_ARCH=mipseb DEFAULT +MACHINE=shark MACHINE_ARCH=arm ALIAS=oshark +MACHINE=shark MACHINE_ARCH=earmv4 ALIAS=eshark DEFAULT +MACHINE=sparc MACHINE_ARCH=sparc +MACHINE=sparc64 MACHINE_ARCH=sparc64 +MACHINE=sun2 MACHINE_ARCH=m68000 +MACHINE=sun3 MACHINE_ARCH=m68k +MACHINE=vax MACHINE_ARCH=vax +MACHINE=x68k MACHINE_ARCH=m68k +MACHINE=zaurus MACHINE_ARCH=arm ALIAS=ozaurus +MACHINE=zaurus MACHINE_ARCH=earm ALIAS=ezaurus DEFAULT +' + +# getarch -- find the default MACHINE_ARCH for a MACHINE, +# or convert an alias to a MACHINE/MACHINE_ARCH pair. +# +# Saves the original value of MACHINE in makewrappermachine before +# alias processing. +# +# Sets MACHINE and MACHINE_ARCH if the input MACHINE value is +# recognised as an alias, or recognised as a machine that has a default +# MACHINE_ARCH (or that has only one possible MACHINE_ARCH). +# +# Leaves MACHINE and MACHINE_ARCH unchanged if MACHINE is recognised +# as being associated with multiple MACHINE_ARCH values with no default. +# +# Bombs if MACHINE is not recognised. +# +getarch() +{ + local IFS + local found="" + local line + + IFS="${nl}" + makewrappermachine="${MACHINE}" + for line in ${valid_MACHINE_ARCH}; do + line="${line%%#*}" # ignore comments + line="$( IFS=" ${tab}" ; echo $line )" # normalise white space + case "${line} " in + " ") + # skip blank lines or comment lines + continue + ;; + *" ALIAS=${MACHINE} "*) + # Found a line with a matching ALIAS=. + found="$line" + break + ;; + "MACHINE=${MACHINE} "*" NO_DEFAULT"*) + # Found an explicit "NO_DEFAULT" for this MACHINE. + found="$line" + break + ;; + "MACHINE=${MACHINE} "*" DEFAULT"*) + # Found an explicit "DEFAULT" for this MACHINE. + found="$line" + break + ;; + "MACHINE=${MACHINE} "*) + # Found a line for this MACHINE. If it's the + # first such line, then tentatively accept it. + # If it's not the first matching line, then + # remember that there was more than one match. + case "$found" in + '') found="$line" ;; + *) found="MULTIPLE_MATCHES" ;; + esac + ;; + esac + done + + case "$found" in + *NO_DEFAULT*|*MULTIPLE_MATCHES*) + # MACHINE is OK, but MACHINE_ARCH is still unknown + return + ;; + "MACHINE="*" MACHINE_ARCH="*) + # Obey the MACHINE= and MACHINE_ARCH= parts of the line. + IFS=" " + for frag in ${found}; do + case "$frag" in + MACHINE=*|MACHINE_ARCH=*) + eval "$frag" + ;; + esac + done + ;; + *) + bomb "Unknown target MACHINE: ${MACHINE}" + ;; + esac +} + +# validatearch -- check that the MACHINE/MACHINE_ARCH pair is supported. +# +# Bombs if the pair is not supported. +# +validatearch() +{ + local IFS + local line + local foundpair=false foundmachine=false foundarch=false + + case "${MACHINE_ARCH}" in + "") + bomb "No MACHINE_ARCH provided" + ;; + esac + + IFS="${nl}" + for line in ${valid_MACHINE_ARCH}; do + line="${line%%#*}" # ignore comments + line="$( IFS=" ${tab}" ; echo $line )" # normalise white space + case "${line} " in + " ") + # skip blank lines or comment lines + continue + ;; + "MACHINE=${MACHINE} MACHINE_ARCH=${MACHINE_ARCH} "*) + foundpair=true + ;; + "MACHINE=${MACHINE} "*) + foundmachine=true + ;; + *"MACHINE_ARCH=${MACHINE_ARCH} "*) + foundarch=true + ;; + esac + done + + case "${foundpair}:${foundmachine}:${foundarch}" in + true:*) + : OK + ;; + *:false:*) + bomb "Unknown target MACHINE: ${MACHINE}" + ;; + *:*:false) + bomb "Unknown target MACHINE_ARCH: ${MACHINE_ARCH}" + ;; + *) + bomb "MACHINE_ARCH '${MACHINE_ARCH}' does not support MACHINE '${MACHINE}'" + ;; + esac +} + +# listarch -- list valid MACHINE/MACHINE_ARCH/ALIAS values, +# optionally restricted to those where the MACHINE and/or MACHINE_ARCH +# match specifed glob patterns. +# +listarch() +{ + local machglob="$1" archglob="$2" + local IFS + local wildcard="*" + local line xline frag + local line_matches_machine line_matches_arch + local found=false + + # Empty machglob or archglob should match anything + : "${machglob:=${wildcard}}" + : "${archglob:=${wildcard}}" + + IFS="${nl}" + for line in ${valid_MACHINE_ARCH}; do + line="${line%%#*}" # ignore comments + xline="$( IFS=" ${tab}" ; echo $line )" # normalise white space + [ -z "${xline}" ] && continue # skip blank or comment lines + + line_matches_machine=false + line_matches_arch=false + + IFS=" " + for frag in ${xline}; do + case "${frag}" in + MACHINE=${machglob}) + line_matches_machine=true ;; + ALIAS=${machglob}) + line_matches_machine=true ;; + MACHINE_ARCH=${archglob}) + line_matches_arch=true ;; + esac + done + + if $line_matches_machine && $line_matches_arch; then + found=true + echo "$line" + fi + done + if ! $found; then + echo >&2 "No match for" \ + "MACHINE=${machglob} MACHINE_ARCH=${archglob}" + return 1 + fi + return 0 +} + +# nobomb_getmakevar -- +# Given the name of a make variable in $1, print make's idea of the +# value of that variable, or return 1 if there's an error. +# +nobomb_getmakevar() +{ + [ -x "${make}" ] || return 1 + "${make}" -m ${TOP}/share/mk -s -B -f- _x_ < +.include +EOF +} + +# bomb_getmakevar -- +# Given the name of a make variable in $1, print make's idea of the +# value of that variable, or bomb if there's an error. +# +bomb_getmakevar() +{ + [ -x "${make}" ] || bomb "bomb_getmakevar $1: ${make} is not executable" + nobomb_getmakevar "$1" || bomb "bomb_getmakevar $1: ${make} failed" +} + +# getmakevar -- +# Given the name of a make variable in $1, print make's idea of the +# value of that variable, or print a literal '$' followed by the +# variable name if ${make} is not executable. This is intended for use in +# messages that need to be readable even if $make hasn't been built, +# such as when build.sh is run with the "-n" option. +# +getmakevar() +{ + if [ -x "${make}" ]; then + bomb_getmakevar "$1" + else + echo "\$$1" + fi +} + +setmakeenv() +{ + eval "$1='$2'; export $1" + makeenv="${makeenv} $1" +} +safe_setmakeenv() +{ + case "$1" in + + # Look for any vars we want to prohibit here, like: + # Bad | Dangerous) usage "Cannot override $1 with -V";; + + # That first char is OK has already been verified. + *[!A-Za-z0-9_]*) usage "Bad variable name (-V): '$1'";; + esac + setmakeenv "$@" +} + +unsetmakeenv() +{ + eval "unset $1" + makeenv="${makeenv} $1" +} +safe_unsetmakeenv() +{ + case "$1" in + + # Look for any vars user should not be able to unset + # Needed | Must_Have) usage "Variable $1 cannot be unset";; + + [!A-Za-z_]* | *[!A-Za-z0-9_]*) usage "Bad variable name (-Z): '$1'";; + esac + unsetmakeenv "$1" +} + +# Given a variable name in $1, modify the variable in place as follows: +# For each space-separated word in the variable, call resolvepath. +resolvepaths() +{ + local var="$1" + local val + eval val=\"\${${var}}\" + local newval='' + local word + for word in ${val}; do + resolvepath word + newval="${newval}${newval:+ }${word}" + done + eval ${var}=\"\${newval}\" +} + +# Given a variable name in $1, modify the variable in place as follows: +# Convert possibly-relative path to absolute path by prepending +# ${TOP} if necessary. Also delete trailing "/", if any. +resolvepath() +{ + local var="$1" + local val + eval val=\"\${${var}}\" + case "${val}" in + /) + ;; + /*) + val="${val%/}" + ;; + *) + val="${TOP}/${val%/}" + ;; + esac + eval ${var}=\"\${val}\" +} + +usage() +{ + if [ -n "$*" ]; then + echo "" + echo "${progname}: $*" + fi + cat <<_usage_ + +Usage: ${progname} [-EhnoPRrUuxy] [-a arch] [-B buildid] [-C cdextras] + [-D dest] [-j njob] [-M obj] [-m mach] [-N noisy] + [-O obj] [-R release] [-S seed] [-T tools] + [-V var=[value]] [-w wrapper] [-X x11src] [-Y extsrcsrc] + [-Z var] + operation [...] + + Build operations (all imply "obj" and "tools"): + build Run "make build". + distribution Run "make distribution" (includes DESTDIR/etc/ files). + release Run "make release" (includes kernels & distrib media). + + Other operations: + help Show this message and exit. + makewrapper Create ${toolprefix}make-\${MACHINE} wrapper and ${toolprefix}make. + Always performed. + cleandir Run "make cleandir". [Default unless -u is used] + obj Run "make obj". [Default unless -o is used] + tools Build and install tools. + install=idir Run "make installworld" to \`idir' to install all sets + except \`etc'. Useful after "distribution" or "release" + kernel=conf Build kernel with config file \`conf' + kernel.gdb=conf Build kernel (including netbsd.gdb) with config + file \`conf' + releasekernel=conf Install kernel built by kernel=conf to RELEASEDIR. + kernels Build all kernels + installmodules=idir Run "make installmodules" to \`idir' to install all + kernel modules. + modules Build kernel modules. + rumptest Do a linktest for rump (for developers). + sets Create binary sets in + RELEASEDIR/RELEASEMACHINEDIR/binary/sets. + DESTDIR should be populated beforehand. + sourcesets Create source sets in RELEASEDIR/source/sets. + syspkgs Create syspkgs in + RELEASEDIR/RELEASEMACHINEDIR/binary/syspkgs. + iso-image Create CD-ROM image in RELEASEDIR/images. + iso-image-source Create CD-ROM image with source in RELEASEDIR/images. + live-image Create bootable live image in + RELEASEDIR/RELEASEMACHINEDIR/installation/liveimage. + install-image Create bootable installation image in + RELEASEDIR/RELEASEMACHINEDIR/installation/installimage. + disk-image=target Create bootable disk image in + RELEASEDIR/RELEASEMACHINEDIR/binary/gzimg/target.img.gz. + params Display various make(1) parameters. + list-arch Display a list of valid MACHINE/MACHINE_ARCH values, + and exit. The list may be narrowed by passing glob + patterns or exact values in MACHINE or MACHINE_ARCH. + + Options: + -a arch Set MACHINE_ARCH to arch. [Default: deduced from MACHINE] + -B buildid Set BUILDID to buildid. + -C cdextras Append cdextras to CDEXTRA variable for inclusion on CD-ROM. + -D dest Set DESTDIR to dest. [Default: destdir.MACHINE] + -E Set "expert" mode; disables various safety checks. + Should not be used without expert knowledge of the build system. + -h Print this help message. + -j njob Run up to njob jobs in parallel; see make(1) -j. + -M obj Set obj root directory to obj; sets MAKEOBJDIRPREFIX. + Unsets MAKEOBJDIR. + -m mach Set MACHINE to mach. Some mach values are actually + aliases that set MACHINE/MACHINE_ARCH pairs. + [Default: deduced from the host system if the host + OS is NetBSD] + -N noisy Set the noisyness (MAKEVERBOSE) level of the build: + 0 Minimal output ("quiet") + 1 Describe what is occurring + 2 Describe what is occurring and echo the actual command + 3 Ignore the effect of the "@" prefix in make commands + 4 Trace shell commands using the shell's -x flag + [Default: 2] + -n Show commands that would be executed, but do not execute them. + -O obj Set obj root directory to obj; sets a MAKEOBJDIR pattern. + Unsets MAKEOBJDIRPREFIX. + -o Set MKOBJDIRS=no; do not create objdirs at start of build. + -P Set MKREPRO and MKREPRO_TIMESTAMP to the latest source + CVS timestamp for reproducible builds. + -R release Set RELEASEDIR to release. [Default: releasedir] + -r Remove contents of TOOLDIR and DESTDIR before building. + -S seed Set BUILDSEED to seed. [Default: NetBSD-majorversion] + -T tools Set TOOLDIR to tools. If unset, and TOOLDIR is not set in + the environment, ${toolprefix}make will be (re)built + unconditionally. + -U Set MKUNPRIVED=yes; build without requiring root privileges, + install from an UNPRIVED build with proper file permissions. + -u Set MKUPDATE=yes; do not run "make cleandir" first. + Without this, everything is rebuilt, including the tools. + -V var=[value] Set variable \`var' to \`value'. + -w wrapper Create ${toolprefix}make script as wrapper. + [Default: \${TOOLDIR}/bin/${toolprefix}make-\${MACHINE}] + -X x11src Set X11SRCDIR to x11src. [Default: /usr/xsrc] + -x Set MKX11=yes; build X11 from X11SRCDIR + -Y extsrcsrc Set EXTSRCSRCDIR to extsrcsrc. [Default: /usr/extsrc] + -y Set MKEXTSRC=yes; build extsrc from EXTSRCSRCDIR + -Z var Unset ("zap") variable \`var'. + +_usage_ + exit 1 +} + +parseoptions() +{ + opts='a:B:C:D:Ehj:M:m:N:nO:oPR:rS:T:UuV:w:X:xY:yZ:' + opt_a=false + opt_m=false + + if type getopts >/dev/null 2>&1; then + # Use POSIX getopts. + # + getoptcmd='getopts ${opts} opt && opt=-${opt}' + optargcmd=':' + optremcmd='shift $((${OPTIND} -1))' + else + type getopt >/dev/null 2>&1 || + bomb "Shell does not support getopts or getopt" + + # Use old-style getopt(1) (doesn't handle whitespace in args). + # + args="$(getopt ${opts} $*)" + [ $? = 0 ] || usage + set -- ${args} + + getoptcmd='[ $# -gt 0 ] && opt="$1" && shift' + optargcmd='OPTARG="$1"; shift' + optremcmd=':' + fi + + # Parse command line options. + # + while eval ${getoptcmd}; do + case ${opt} in + + -a) + eval ${optargcmd} + MACHINE_ARCH=${OPTARG} + opt_a=true + ;; + + -B) + eval ${optargcmd} + BUILDID=${OPTARG} + ;; + + -C) + eval ${optargcmd}; resolvepaths OPTARG + CDEXTRA="${CDEXTRA}${CDEXTRA:+ }${OPTARG}" + ;; + + -D) + eval ${optargcmd}; resolvepath OPTARG + setmakeenv DESTDIR "${OPTARG}" + ;; + + -E) + do_expertmode=true + ;; + + -j) + eval ${optargcmd} + parallel="-j ${OPTARG}" + ;; + + -M) + eval ${optargcmd}; resolvepath OPTARG + case "${OPTARG}" in + \$*) usage "-M argument must not begin with '\$'" + ;; + *\$*) # can use resolvepath, but can't set TOP_objdir + resolvepath OPTARG + ;; + *) resolvepath OPTARG + TOP_objdir="${OPTARG}${TOP}" + ;; + esac + unsetmakeenv MAKEOBJDIR + setmakeenv MAKEOBJDIRPREFIX "${OPTARG}" + ;; + + # -m overrides MACHINE_ARCH unless "-a" is specified + -m) + eval ${optargcmd} + MACHINE="${OPTARG}" + opt_m=true + ;; + + -N) + eval ${optargcmd} + case "${OPTARG}" in + 0|1|2|3|4) + setmakeenv MAKEVERBOSE "${OPTARG}" + ;; + *) + usage "'${OPTARG}' is not a valid value for -N" + ;; + esac + ;; + + -n) + runcmd=echo + ;; + + -O) + eval ${optargcmd} + case "${OPTARG}" in + *\$*) usage "-O argument must not contain '\$'" + ;; + *) resolvepath OPTARG + TOP_objdir="${OPTARG}" + ;; + esac + unsetmakeenv MAKEOBJDIRPREFIX + setmakeenv MAKEOBJDIR "\${.CURDIR:C,^$TOP,$OPTARG,}" + ;; + + -o) + MKOBJDIRS=no + ;; + + -P) + MKREPRO=yes + ;; + + -R) + eval ${optargcmd}; resolvepath OPTARG + setmakeenv RELEASEDIR "${OPTARG}" + ;; + + -r) + do_removedirs=true + do_rebuildmake=true + ;; + + -S) + eval ${optargcmd} + setmakeenv BUILDSEED "${OPTARG}" + ;; + + -T) + eval ${optargcmd}; resolvepath OPTARG + TOOLDIR="${OPTARG}" + export TOOLDIR + ;; + + -U) + setmakeenv MKUNPRIVED yes + ;; + + -u) + setmakeenv MKUPDATE yes + ;; + + -V) + eval ${optargcmd} + case "${OPTARG}" in + # XXX: consider restricting which variables can be changed? + [a-zA-Z_]*=*) + safe_setmakeenv "${OPTARG%%=*}" "${OPTARG#*=}" + ;; + [a-zA-Z_]*) + safe_setmakeenv "${OPTARG}" "" + ;; + *) + usage "-V argument must be of the form 'var=[value]'" + ;; + esac + ;; + + -w) + eval ${optargcmd}; resolvepath OPTARG + makewrapper="${OPTARG}" + ;; + + -X) + eval ${optargcmd}; resolvepath OPTARG + setmakeenv X11SRCDIR "${OPTARG}" + ;; + + -x) + setmakeenv MKX11 yes + ;; + + -Y) + eval ${optargcmd}; resolvepath OPTARG + setmakeenv EXTSRCSRCDIR "${OPTARG}" + ;; + + -y) + setmakeenv MKEXTSRC yes + ;; + + -Z) + eval ${optargcmd} + # XXX: consider restricting which variables can be unset? + safe_unsetmakeenv "${OPTARG}" + ;; + + --) + break + ;; + + -'?'|-h) + usage + ;; + + esac + done + + # Validate operations. + # + eval ${optremcmd} + while [ $# -gt 0 ]; do + op=$1; shift + operations="${operations} ${op}" + + case "${op}" in + + help) + usage + ;; + + list-arch) + listarch "${MACHINE}" "${MACHINE_ARCH}" + exit $? + ;; + + kernel=*|releasekernel=*|kernel.gdb=*) + arg=${op#*=} + op=${op%%=*} + [ -n "${arg}" ] || + bomb "Must supply a kernel name with \`${op}=...'" + ;; + + disk-image=*) + arg=${op#*=} + op=disk_image + [ -n "${arg}" ] || + bomb "Must supply a target name with \`${op}=...'" + + ;; + + install=*|installmodules=*) + arg=${op#*=} + op=${op%%=*} + [ -n "${arg}" ] || + bomb "Must supply a directory with \`install=...'" + ;; + + build|\ + cleandir|\ + distribution|\ + install-image|\ + iso-image-source|\ + iso-image|\ + kernels|\ + live-image|\ + makewrapper|\ + modules|\ + obj|\ + params|\ + release|\ + rump|\ + rumptest|\ + sets|\ + sourcesets|\ + syspkgs|\ + tools) + ;; + + *) + usage "Unknown operation \`${op}'" + ;; + + esac + # ${op} may contain chars that are not allowed in variable + # names. Replace them with '_' before setting do_${op}. + op="$( echo "$op" | tr -s '.-' '__')" + eval do_${op}=true + done + [ -n "${operations}" ] || usage "Missing operation to perform." + + # Set up MACHINE*. On a NetBSD host, these are allowed to be unset. + # + if [ -z "${MACHINE}" ]; then + [ "${uname_s}" = "NetBSD" ] || + bomb "MACHINE must be set, or -m must be used, for cross builds." + MACHINE=${uname_m} + fi + if $opt_m && ! $opt_a; then + # Settings implied by the command line -m option + # override MACHINE_ARCH from the environment (if any). + getarch + fi + [ -n "${MACHINE_ARCH}" ] || getarch + validatearch + + # Set up default make(1) environment. + # + makeenv="${makeenv} TOOLDIR MACHINE MACHINE_ARCH MAKEFLAGS" + [ -z "${BUILDID}" ] || makeenv="${makeenv} BUILDID" + [ -z "${BUILDINFO}" ] || makeenv="${makeenv} BUILDINFO" + MAKEFLAGS="-de -m ${TOP}/share/mk ${MAKEFLAGS}" + MAKEFLAGS="${MAKEFLAGS} MKOBJDIRS=${MKOBJDIRS-yes}" + export MAKEFLAGS MACHINE MACHINE_ARCH + setmakeenv USETOOLS "yes" + setmakeenv MAKEWRAPPERMACHINE "${makewrappermachine:-${MACHINE}}" +} + +# sanitycheck -- +# Sanity check after parsing command line options, before rebuildmake. +# +sanitycheck() +{ + # Install as non-root is a bad idea. + # + if ${do_install} && [ "$id_u" -ne 0 ] ; then + if ${do_expertmode}; then + warning "Will install as an unprivileged user." + else + bomb "-E must be set for install as an unprivileged user." + fi + fi + + # If the PATH contains any non-absolute components (including, + # but not limited to, "." or ""), then complain. As an exception, + # allow "" or "." as the last component of the PATH. This is fatal + # if expert mode is not in effect. + # + local path="${PATH}" + path="${path%:}" # delete trailing ":" + path="${path%:.}" # delete trailing ":." + case ":${path}:/" in + *:[!/]*) + if ${do_expertmode}; then + warning "PATH contains non-absolute components" + else + bomb "PATH environment variable must not" \ + "contain non-absolute components" + fi + ;; + esac + + while [ ${MKX11-no} = "yes" ]; do # not really a loop + test -n "${X11SRCDIR}" && { + test -d "${X11SRCDIR}" || + bomb "X11SRCDIR (${X11SRCDIR}) does not exist (with -x)" + break + } + for _xd in \ + "${NETBSDSRCDIR%/*}/xsrc" \ + "${NETBSDSRCDIR}/xsrc" \ + /usr/xsrc + do + test -d "${_xd}" && + setmakeenv X11SRCDIR "${_xd}" && + break 2 + done + bomb "Asked to build X11 but no xsrc" + done +} + +# print_tooldir_make -- +# Try to find and print a path to an existing +# ${TOOLDIR}/bin/${toolprefix}program +print_tooldir_program() +{ + local possible_TOP_OBJ + local possible_TOOLDIR + local possible_program + local tooldir_program + local program=${1} + + if [ -n "${TOOLDIR}" ]; then + echo "${TOOLDIR}/bin/${toolprefix}${program}" + return + fi + + # Set host_ostype to something like "NetBSD-4.5.6-i386". This + # is intended to match the HOST_OSTYPE variable in . + # + local host_ostype="${uname_s}-$( + echo "${uname_r}" | sed -e 's/([^)]*)//g' -e 's/ /_/g' + )-$( + echo "${uname_p}" | sed -e 's/([^)]*)//g' -e 's/ /_/g' + )" + + # Look in a few potential locations for + # ${possible_TOOLDIR}/bin/${toolprefix}${program}. + # If we find it, then set possible_program. + # + # In the usual case (without interference from environment + # variables or /etc/mk.conf), should set TOOLDIR to + # "${_SRC_TOP_OBJ_}/tooldir.${host_ostype}". + # + # In practice it's difficult to figure out the correct value + # for _SRC_TOP_OBJ_. In the easiest case, when the -M or -O + # options were passed to build.sh, then ${TOP_objdir} will be + # the correct value. We also try a few other possibilities, but + # we do not replicate all the logic of . + # + for possible_TOP_OBJ in \ + "${TOP_objdir}" \ + "${MAKEOBJDIRPREFIX:+${MAKEOBJDIRPREFIX}${TOP}}" \ + "${TOP}" \ + "${TOP}/obj" \ + "${TOP}/obj.${MACHINE}" + do + [ -n "${possible_TOP_OBJ}" ] || continue + possible_TOOLDIR="${possible_TOP_OBJ}/tooldir.${host_ostype}" + possible_program="${possible_TOOLDIR}/bin/${toolprefix}${program}" + if [ -x "${possible_make}" ]; then + echo ${possible_program} + return; + fi + done + echo "" +} +# print_tooldir_make -- +# Try to find and print a path to an existing +# ${TOOLDIR}/bin/${toolprefix}make, for use by rebuildmake() before a +# new version of ${toolprefix}make has been built. +# +# * If TOOLDIR was set in the environment or on the command line, use +# that value. +# * Otherwise try to guess what TOOLDIR would be if not overridden by +# /etc/mk.conf, and check whether the resulting directory contains +# a copy of ${toolprefix}make (this should work for everybody who +# doesn't override TOOLDIR via /etc/mk.conf); +# * Failing that, search for ${toolprefix}make, nbmake, bmake, or make, +# in the PATH (this might accidentally find a version of make that +# does not understand the syntax used by NetBSD make, and that will +# lead to failure in the next step); +# * If a copy of make was found above, try to use it with +# nobomb_getmakevar to find the correct value for TOOLDIR, and believe the +# result only if it's a directory that already exists; +# * If a value of TOOLDIR was found above, and if +# ${TOOLDIR}/bin/${toolprefix}make exists, print that value. +# +print_tooldir_make() +{ + local possible_make + local possible_TOOLDIR + local tooldir_make + + possible_make=$(print_tooldir_program make) + # If the above didn't work, search the PATH for a suitable + # ${toolprefix}make, nbmake, bmake, or make. + # + : ${possible_make:=$(find_in_PATH ${toolprefix}make '')} + : ${possible_make:=$(find_in_PATH nbmake '')} + : ${possible_make:=$(find_in_PATH bmake '')} + : ${possible_make:=$(find_in_PATH make '')} + + # At this point, we don't care whether possible_make is in the + # correct TOOLDIR or not; we simply want it to be usable by + # getmakevar to help us find the correct TOOLDIR. + # + # Use ${possible_make} with nobomb_getmakevar to try to find + # the value of TOOLDIR. Believe the result only if it's + # a directory that already exists and contains bin/${toolprefix}make. + # + if [ -x "${possible_make}" ]; then + possible_TOOLDIR="$( + make="${possible_make}" \ + nobomb_getmakevar TOOLDIR 2>/dev/null + )" + if [ $? = 0 ] && [ -n "${possible_TOOLDIR}" ] \ + && [ -d "${possible_TOOLDIR}" ]; + then + tooldir_make="${possible_TOOLDIR}/bin/${toolprefix}make" + if [ -x "${tooldir_make}" ]; then + echo "${tooldir_make}" + return 0 + fi + fi + fi + return 1 +} + +# rebuildmake -- +# Rebuild nbmake in a temporary directory if necessary. Sets $make +# to a path to the nbmake executable. Sets done_rebuildmake=true +# if nbmake was rebuilt. +# +# There is a cyclic dependency between building nbmake and choosing +# TOOLDIR: TOOLDIR may be affected by settings in /etc/mk.conf, so we +# would like to use getmakevar to get the value of TOOLDIR; but we can't +# use getmakevar before we have an up to date version of nbmake; we +# might already have an up to date version of nbmake in TOOLDIR, but we +# don't yet know where TOOLDIR is. +# +# The default value of TOOLDIR also depends on the location of the top +# level object directory, so $(getmakevar TOOLDIR) invoked before or +# after making the top level object directory may produce different +# results. +# +# Strictly speaking, we should do the following: +# +# 1. build a new version of nbmake in a temporary directory; +# 2. use the temporary nbmake to create the top level obj directory; +# 3. use $(getmakevar TOOLDIR) with the temporary nbmake to +# get the correct value of TOOLDIR; +# 4. move the temporary nbmake to ${TOOLDIR}/bin/nbmake. +# +# However, people don't like building nbmake unnecessarily if their +# TOOLDIR has not changed since an earlier build. We try to avoid +# rebuilding a temporary version of nbmake by taking some shortcuts to +# guess a value for TOOLDIR, looking for an existing version of nbmake +# in that TOOLDIR, and checking whether that nbmake is newer than the +# sources used to build it. +# +rebuildmake() +{ + make="$(print_tooldir_make)" + if [ -n "${make}" ] && [ -x "${make}" ]; then + for f in usr.bin/make/*.[ch] usr.bin/make/lst.lib/*.[ch]; do + if [ "${f}" -nt "${make}" ]; then + statusmsg "${make} outdated" \ + "(older than ${f}), needs building." + do_rebuildmake=true + break + fi + done + else + statusmsg "No \$TOOLDIR/bin/${toolprefix}make, needs building." + do_rebuildmake=true + fi + + # Build bootstrap ${toolprefix}make if needed. + if ! ${do_rebuildmake}; then + return + fi + + statusmsg "Bootstrapping ${toolprefix}make" + ${runcmd} cd "${tmpdir}" + ${runcmd} env CC="${HOST_CC-cc}" CPPFLAGS="${HOST_CPPFLAGS}" \ + CFLAGS="${HOST_CFLAGS--O}" LDFLAGS="${HOST_LDFLAGS}" \ + ${HOST_SH} "${TOP}/tools/make/configure" || + ( cp ${tmpdir}/config.log ${tmpdir}-config.log + bomb "Configure of ${toolprefix}make failed, see ${tmpdir}-config.log for details" ) + ${runcmd} ${HOST_SH} buildmake.sh || + bomb "Build of ${toolprefix}make failed" + make="${tmpdir}/${toolprefix}make" + ${runcmd} cd "${TOP}" + ${runcmd} rm -f usr.bin/make/*.o usr.bin/make/lst.lib/*.o + done_rebuildmake=true +} + +# validatemakeparams -- +# Perform some late sanity checks, after rebuildmake, +# but before createmakewrapper or any real work. +# +# Creates the top-level obj directory, because that +# is needed by some of the sanity checks. +# +# Prints status messages reporting the values of several variables. +# +validatemakeparams() +{ + # MAKECONF (which defaults to /etc/mk.conf in share/mk/bsd.own.mk) + # can affect many things, so mention it in an early status message. + # + MAKECONF=$(getmakevar MAKECONF) + if [ -e "${MAKECONF}" ]; then + statusmsg2 "MAKECONF file:" "${MAKECONF}" + else + statusmsg2 "MAKECONF file:" "${MAKECONF} (File not found)" + fi + + # Normalise MKOBJDIRS, MKUNPRIVED, and MKUPDATE. + # These may be set as build.sh options or in "mk.conf". + # Don't export them as they're only used for tests in build.sh. + # + MKOBJDIRS=$(getmakevar MKOBJDIRS) + MKUNPRIVED=$(getmakevar MKUNPRIVED) + MKUPDATE=$(getmakevar MKUPDATE) + + # Non-root should always use either the -U or -E flag. + # + if ! ${do_expertmode} && \ + [ "$id_u" -ne 0 ] && \ + [ "${MKUNPRIVED}" = "no" ] ; then + bomb "-U or -E must be set for build as an unprivileged user." + fi + + if [ "${runcmd}" = "echo" ]; then + TOOLCHAIN_MISSING=no + EXTERNAL_TOOLCHAIN="" + else + TOOLCHAIN_MISSING=$(bomb_getmakevar TOOLCHAIN_MISSING) + EXTERNAL_TOOLCHAIN=$(bomb_getmakevar EXTERNAL_TOOLCHAIN) + fi + if [ "${TOOLCHAIN_MISSING}" = "yes" ] && \ + [ -z "${EXTERNAL_TOOLCHAIN}" ]; then + ${runcmd} echo "ERROR: build.sh (in-tree cross-toolchain) is not yet available for" + ${runcmd} echo " MACHINE: ${MACHINE}" + ${runcmd} echo " MACHINE_ARCH: ${MACHINE_ARCH}" + ${runcmd} echo "" + ${runcmd} echo "All builds for this platform should be done via a traditional make" + ${runcmd} echo "If you wish to use an external cross-toolchain, set" + ${runcmd} echo " EXTERNAL_TOOLCHAIN=" + ${runcmd} echo "in either the environment or mk.conf and rerun" + ${runcmd} echo " ${progname} $*" + exit 1 + fi + + if [ "${MKOBJDIRS}" != "no" ]; then + # Create the top-level object directory. + # + # "make obj NOSUBDIR=" can handle most cases, but it + # can't handle the case where MAKEOBJDIRPREFIX is set + # while the corresponding directory does not exist + # (rules in would abort the build). We + # therefore have to handle the MAKEOBJDIRPREFIX case + # without invoking "make obj". The MAKEOBJDIR case + # could be handled either way, but we choose to handle + # it similarly to MAKEOBJDIRPREFIX. + # + if [ -n "${TOP_obj}" ]; then + # It must have been set by the "-M" or "-O" + # command line options, so there's no need to + # use getmakevar + : + elif [ -n "$MAKEOBJDIRPREFIX" ]; then + TOP_obj="$(getmakevar MAKEOBJDIRPREFIX)${TOP}" + elif [ -n "$MAKEOBJDIR" ]; then + TOP_obj="$(getmakevar MAKEOBJDIR)" + fi + if [ -n "$TOP_obj" ]; then + ${runcmd} mkdir -p "${TOP_obj}" || + bomb "Can't create top level object directory" \ + "${TOP_obj}" + else + ${runcmd} "${make}" -m ${TOP}/share/mk obj NOSUBDIR= || + bomb "Can't create top level object directory" \ + "using make obj" + fi + + # make obj in tools to ensure that the objdir for "tools" + # is available. + # + ${runcmd} cd tools + ${runcmd} "${make}" -m ${TOP}/share/mk obj NOSUBDIR= || + bomb "Failed to make obj in tools" + ${runcmd} cd "${TOP}" + fi + + # Find TOOLDIR, DESTDIR, and RELEASEDIR, according to getmakevar, + # and bomb if they have changed from the values we had from the + # command line or environment. + # + # This must be done after creating the top-level object directory. + # + for var in TOOLDIR DESTDIR RELEASEDIR + do + eval oldval=\"\$${var}\" + newval="$(getmakevar $var)" + if ! $do_expertmode; then + : ${_SRC_TOP_OBJ_:=$(getmakevar _SRC_TOP_OBJ_)} + case "$var" in + DESTDIR) + : ${newval:=${_SRC_TOP_OBJ_}/destdir.${MACHINE}} + makeenv="${makeenv} DESTDIR" + ;; + RELEASEDIR) + : ${newval:=${_SRC_TOP_OBJ_}/releasedir} + makeenv="${makeenv} RELEASEDIR" + ;; + esac + fi + if [ -n "$oldval" ] && [ "$oldval" != "$newval" ]; then + bomb "Value of ${var} has changed" \ + "(was \"${oldval}\", now \"${newval}\")" + fi + eval ${var}=\"\${newval}\" + eval export ${var} + statusmsg2 "${var} path:" "${newval}" + done + + # RELEASEMACHINEDIR is just a subdir name, e.g. "i386". + RELEASEMACHINEDIR=$(getmakevar RELEASEMACHINEDIR) + + # Check validity of TOOLDIR and DESTDIR. + # + if [ -z "${TOOLDIR}" ] || [ "${TOOLDIR}" = "/" ]; then + bomb "TOOLDIR '${TOOLDIR}' invalid" + fi + removedirs="${TOOLDIR}" + + if [ -z "${DESTDIR}" ] || [ "${DESTDIR}" = "/" ]; then + if ${do_distribution} || ${do_release} || \ + [ "${uname_s}" != "NetBSD" ] || \ + [ "${uname_m}" != "${MACHINE}" ]; then + bomb "DESTDIR must != / for cross builds, or ${progname} 'distribution' or 'release'." + fi + if ! ${do_expertmode}; then + bomb "DESTDIR must != / for non -E (expert) builds" + fi + statusmsg "WARNING: Building to /, in expert mode." + statusmsg " This may cause your system to break! Reasons include:" + statusmsg " - your kernel is not up to date" + statusmsg " - the libraries or toolchain have changed" + statusmsg " YOU HAVE BEEN WARNED!" + else + removedirs="${removedirs} ${DESTDIR}" + fi + if ${do_releasekernel} && [ -z "${RELEASEDIR}" ]; then + bomb "Must set RELEASEDIR with \`releasekernel=...'" + fi + + # If a previous build.sh run used -U (and therefore created a + # METALOG file), then most subsequent build.sh runs must also + # use -U. If DESTDIR is about to be removed, then don't perform + # this check. + # + case "${do_removedirs} ${removedirs} " in + true*" ${DESTDIR} "*) + # DESTDIR is about to be removed + ;; + *) + if [ -e "${DESTDIR}/METALOG" ] && \ + [ "${MKUNPRIVED}" = "no" ] ; then + if $do_expertmode; then + warning "A previous build.sh run specified -U." + else + bomb "A previous build.sh run specified -U; you must specify it again now." + fi + fi + ;; + esac + + # live-image and install-image targets require binary sets + # (actually DESTDIR/etc/mtree/set.* files) built with MKUNPRIVED. + # If release operation is specified with live-image or install-image, + # the release op should be performed with -U for later image ops. + # + if ${do_release} && ( ${do_live_image} || ${do_install_image} ) && \ + [ "${MKUNPRIVED}" = "no" ] ; then + bomb "-U must be specified on building release to create images later." + fi +} + + +createmakewrapper() +{ + # Remove the target directories. + # + if ${do_removedirs}; then + for f in ${removedirs}; do + statusmsg "Removing ${f}" + ${runcmd} rm -r -f "${f}" + done + fi + + # Recreate $TOOLDIR. + # + ${runcmd} mkdir -p "${TOOLDIR}/bin" || + bomb "mkdir of '${TOOLDIR}/bin' failed" + + # If we did not previously rebuild ${toolprefix}make, then + # check whether $make is still valid and the same as the output + # from print_tooldir_make. If not, then rebuild make now. A + # possible reason for this being necessary is that the actual + # value of TOOLDIR might be different from the value guessed + # before the top level obj dir was created. + # + if ! ${done_rebuildmake} && \ + ( [ ! -x "$make" ] || [ "$make" != "$(print_tooldir_make)" ] ) + then + rebuildmake + fi + + # Install ${toolprefix}make if it was built. + # + if ${done_rebuildmake}; then + ${runcmd} rm -f "${TOOLDIR}/bin/${toolprefix}make" + ${runcmd} cp "${make}" "${TOOLDIR}/bin/${toolprefix}make" || + bomb "Failed to install \$TOOLDIR/bin/${toolprefix}make" + make="${TOOLDIR}/bin/${toolprefix}make" + statusmsg "Created ${make}" + fi + + # Build a ${toolprefix}make wrapper script, usable by hand as + # well as by build.sh. + # + if [ -z "${makewrapper}" ]; then + makewrapper="${TOOLDIR}/bin/${toolprefix}make-${makewrappermachine:-${MACHINE}}" + [ -z "${BUILDID}" ] || makewrapper="${makewrapper}-${BUILDID}" + fi + + ${runcmd} rm -f "${makewrapper}" + if [ "${runcmd}" = "echo" ]; then + echo 'cat <'${makewrapper} + makewrapout= + else + makewrapout=">>\${makewrapper}" + fi + + case "${KSH_VERSION:-${SH_VERSION}}" in + *PD\ KSH*|*MIRBSD\ KSH*) + set +o braceexpand + ;; + esac + + eval cat < ${releasekern}" + else + gzip -c -9 < "${builtkern}" > "${releasekern}" + fi + done +} + +buildkernels() +{ + allkernels=$( runcmd= make_in_dir etc '-V ${ALL_KERNELS}' ) + for k in $allkernels; do + buildkernel "${k}" + done +} + +buildmodules() +{ + setmakeenv MKBINUTILS no + if ! ${do_tools} && ! ${buildmoduleswarned:-false}; then + # Building tools every time we build modules is clearly + # unnecessary as well as a kernel. + # + statusmsg "Building modules without building new tools" + buildmoduleswarned=true + fi + + statusmsg "Building kernel modules for NetBSD/${MACHINE} ${DISTRIBVER}" + if [ "${MKOBJDIRS}" != "no" ]; then + make_in_dir sys/modules obj + fi + if [ "${MKUPDATE}" = "no" ]; then + make_in_dir sys/modules cleandir + fi + make_in_dir sys/modules dependall + make_in_dir sys/modules install + + statusmsg "Successful build of kernel modules for NetBSD/${MACHINE} ${DISTRIBVER}" +} + +installmodules() +{ + dir="$1" + ${runcmd} "${makewrapper}" INSTALLMODULESDIR="${dir}" installmodules || + bomb "Failed to make installmodules to ${dir}" + statusmsg "Successful installmodules to ${dir}" +} + +installworld() +{ + dir="$1" + ${runcmd} "${makewrapper}" INSTALLWORLDDIR="${dir}" installworld || + bomb "Failed to make installworld to ${dir}" + statusmsg "Successful installworld to ${dir}" +} + +# Run rump build&link tests. +# +# To make this feasible for running without having to install includes and +# libraries into destdir (i.e. quick), we only run ld. This is possible +# since the rump kernel is a closed namespace apart from calls to rumpuser. +# Therefore, if ld complains only about rumpuser symbols, rump kernel +# linking was successful. +# +# We test that rump links with a number of component configurations. +# These attempt to mimic what is encountered in the full build. +# See list below. The list should probably be either autogenerated +# or managed elsewhere; keep it here until a better idea arises. +# +# Above all, note that THIS IS NOT A SUBSTITUTE FOR A FULL BUILD. +# + +RUMP_LIBSETS=' + -lrump, + -lrumpvfs -lrump, + -lrumpvfs -lrumpdev -lrump, + -lrumpnet -lrump, + -lrumpkern_tty -lrumpvfs -lrump, + -lrumpfs_tmpfs -lrumpvfs -lrump, + -lrumpfs_ffs -lrumpfs_msdos -lrumpvfs -lrumpdev_disk -lrumpdev -lrump, + -lrumpnet_virtif -lrumpnet_netinet -lrumpnet_net -lrumpnet + -lrumpdev -lrumpvfs -lrump, + -lrumpnet_sockin -lrumpfs_smbfs -lrumpdev_netsmb + -lrumpkern_crypto -lrumpdev -lrumpnet -lrumpvfs -lrump, + -lrumpnet_sockin -lrumpfs_nfs -lrumpnet -lrumpvfs -lrump, + -lrumpdev_cgd -lrumpdev_raidframe -lrumpdev_disk -lrumpdev_rnd + -lrumpdev_dm -lrumpdev -lrumpvfs -lrumpkern_crypto -lrump' +dorump() +{ + local doclean="" + local doobjs="" + + # we cannot link libs without building csu, and that leads to lossage + [ "${1}" != "rumptest" ] && bomb 'build.sh rump not yet functional. ' \ + 'did you mean "rumptest"?' + + export RUMPKERN_ONLY=1 + # create obj and distrib dirs + if [ "${MKOBJDIRS}" != "no" ]; then + make_in_dir "${NETBSDSRCDIR}/etc/mtree" obj + make_in_dir "${NETBSDSRCDIR}/sys/rump" obj + fi + ${runcmd} "${makewrapper}" ${parallel} do-distrib-dirs \ + || bomb 'could not create distrib-dirs' + + [ "${MKUPDATE}" = "no" ] && doclean="cleandir" + targlist="${doclean} ${doobjs} dependall install" + # optimize: for test we build only static libs (3x test speedup) + if [ "${1}" = "rumptest" ] ; then + setmakeenv NOPIC 1 + setmakeenv NOPROFILE 1 + fi + for cmd in ${targlist} ; do + make_in_dir "${NETBSDSRCDIR}/sys/rump" ${cmd} + done + + # if we just wanted to build & install rump, we're done + [ "${1}" != "rumptest" ] && return + + ${runcmd} cd "${NETBSDSRCDIR}/sys/rump/librump/rumpkern" \ + || bomb "cd to rumpkern failed" + md_quirks=`${runcmd} "${makewrapper}" -V '${_SYMQUIRK}'` + # one little, two little, three little backslashes ... + md_quirks="$(echo ${md_quirks} | sed 's,\\,\\\\,g'";s/'//g" )" + ${runcmd} cd "${TOP}" || bomb "cd to ${TOP} failed" + tool_ld=`${runcmd} "${makewrapper}" -V '${LD}'` + + local oIFS="${IFS}" + IFS="," + for set in ${RUMP_LIBSETS} ; do + IFS="${oIFS}" + ${runcmd} ${tool_ld} -nostdlib -L${DESTDIR}/usr/lib \ + -static --whole-archive ${set} 2>&1 -o /tmp/rumptest.$$ | \ + awk -v quirks="${md_quirks}" ' + /undefined reference/ && + !/more undefined references.*follow/{ + if (match($NF, + "`(rumpuser_|rumpcomp_|__" quirks ")") == 0) + fails[NR] = $0 + } + /cannot find -l/{fails[NR] = $0} + /cannot open output file/{fails[NR] = $0} + END{ + for (x in fails) + print fails[x] + exit x!=0 + }' + [ $? -ne 0 ] && bomb "Testlink of rump failed: ${set}" + done + statusmsg "Rump build&link tests successful" +} + +setup_mkrepro() +{ + if [ ${MKREPRO-no} != "yes" ]; then + return + fi + local dirs=${NETBSDSRCDIR-/usr/src}/ + if [ ${MKX11-no} = "yes" ]; then + dirs="$dirs ${X11SRCDIR-/usr/xsrc}/" + fi + local cvslatest=$(print_tooldir_program cvslatest) + if [ ! -x "${cvslatest}" ]; then + buildtools + fi + MKREPRO_TIMESTAMP=$("${cvslatest}" ${dirs}) + [ -n "${MKREPRO_TIMESTAMP}" ] || bomb "Failed to compute timestamp" + statusmsg2 "MKREPRO_TIMESTAMP" "$(TZ=UTC date -r ${MKREPRO_TIMESTAMP})" + export MKREPRO MKREPRO_TIMESTAMP +} + +main() +{ + initdefaults + _args=$@ + parseoptions "$@" + + sanitycheck + + build_start=$(date) + statusmsg2 "${progname} command:" "$0 $*" + statusmsg2 "${progname} started:" "${build_start}" + statusmsg2 "NetBSD version:" "${DISTRIBVER}" + statusmsg2 "MACHINE:" "${MACHINE}" + statusmsg2 "MACHINE_ARCH:" "${MACHINE_ARCH}" + statusmsg2 "Build platform:" "${uname_s} ${uname_r} ${uname_m}" + statusmsg2 "HOST_SH:" "${HOST_SH}" + if [ -n "${BUILDID}" ]; then + statusmsg2 "BUILDID:" "${BUILDID}" + fi + if [ -n "${BUILDINFO}" ]; then + printf "%b\n" "${BUILDINFO}" | \ + while read -r line ; do + [ -s "${line}" ] && continue + statusmsg2 "BUILDINFO:" "${line}" + done + fi + + rebuildmake + validatemakeparams + createmakewrapper + setup_mkrepro + + # Perform the operations. + # + for op in ${operations}; do + case "${op}" in + + makewrapper) + # no-op + ;; + + tools) + buildtools + ;; + + sets) + statusmsg "Building sets from pre-populated ${DESTDIR}" + ${runcmd} "${makewrapper}" ${parallel} ${op} || + bomb "Failed to make ${op}" + setdir=${RELEASEDIR}/${RELEASEMACHINEDIR}/binary/sets + statusmsg "Built sets to ${setdir}" + ;; + + build|distribution|release) + ${runcmd} "${makewrapper}" ${parallel} ${op} || + bomb "Failed to make ${op}" + statusmsg "Successful make ${op}" + ;; + + cleandir|obj|sourcesets|syspkgs|params) + ${runcmd} "${makewrapper}" ${parallel} ${op} || + bomb "Failed to make ${op}" + statusmsg "Successful make ${op}" + ;; + + iso-image|iso-image-source) + ${runcmd} "${makewrapper}" ${parallel} \ + CDEXTRA="$CDEXTRA" ${op} || + bomb "Failed to make ${op}" + statusmsg "Successful make ${op}" + ;; + + live-image|install-image) + # install-image and live-image require mtree spec files + # built with UNPRIVED. Assume UNPRIVED build has been + # performed if METALOG file is created in DESTDIR. + if [ ! -e "${DESTDIR}/METALOG" ] ; then + bomb "The release binaries must have been built with -U to create images." + fi + ${runcmd} "${makewrapper}" ${parallel} ${op} || + bomb "Failed to make ${op}" + statusmsg "Successful make ${op}" + ;; + kernel=*) + arg=${op#*=} + buildkernel "${arg}" + ;; + kernel.gdb=*) + arg=${op#*=} + configopts="-D DEBUG=-g" + buildkernel "${arg}" + ;; + releasekernel=*) + arg=${op#*=} + releasekernel "${arg}" + ;; + + kernels) + buildkernels + ;; + + disk-image=*) + arg=${op#*=} + diskimage "${arg}" + ;; + + modules) + buildmodules + ;; + + installmodules=*) + arg=${op#*=} + if [ "${arg}" = "/" ] && \ + ( [ "${uname_s}" != "NetBSD" ] || \ + [ "${uname_m}" != "${MACHINE}" ] ); then + bomb "'${op}' must != / for cross builds." + fi + installmodules "${arg}" + ;; + + install=*) + arg=${op#*=} + if [ "${arg}" = "/" ] && \ + ( [ "${uname_s}" != "NetBSD" ] || \ + [ "${uname_m}" != "${MACHINE}" ] ); then + bomb "'${op}' must != / for cross builds." + fi + installworld "${arg}" + ;; + + rump|rumptest) + dorump "${op}" + ;; + + *) + bomb "Unknown operation \`${op}'" + ;; + + esac + done + + statusmsg2 "${progname} ended:" "$(date)" + if [ -s "${results}" ]; then + echo "===> Summary of results:" + sed -e 's/^===>//;s/^/ /' "${results}" + echo "===> ." + fi +} + +main "$@"