NetBSD/tools
lukem cf9c9d4dc8 tools: revert use of <bsd.hostinit.mk>
tools/Makefile.host revision 1.35 changed from
  setting various NOxxx variables and .include <bsd.own.mk>
to
  .include <bsd.hostinit.mk>
which sets the same NOxxx variables, .include <bsd.init.mk>
which optionally .include ../Makefile.inc before .include <bsd.own.mk>.

This chnage caused build failures with HAVE_LLVM=yes, so revert rev 1.35.
(Maybe to do with the .include ../Makefile.inc. The tools/llvm* build is
tricky, and it's easier to revert this change than debug.)
2023-06-04 16:53:59 +00:00
..
amiga-elf2bb
amiga-txlt
arm-elf2aout s/hidding/hiding/ 2022-05-28 22:00:55 +00:00
asn1_compile
autoconf
awk
binstall Create a new bsd.hostinit.mk file and put the build definitions for all host 2018-05-01 19:59:42 +00:00
binutils remap arm directory names, the same way the source does. 2023-02-07 20:37:30 +00:00
cap_mkdb
cat
cksum
compat regen 2023-06-03 09:18:14 +00:00
compile_et
config
crunchgen
ctags
ctfconvert Update 'tools/ctf{merge,convert}' to use the new build locations for 2022-04-18 19:40:06 +00:00
ctfmerge Update 'tools/ctf{merge,convert}' to use the new build locations for 2022-04-18 19:40:06 +00:00
cvslatest Create a new bsd.hostinit.mk file and put the build definitions for all host 2018-05-01 19:59:42 +00:00
date If we are building date earlier, we need to use an uninstalled copy 2021-05-27 08:41:35 +00:00
db
dbsym Create a new bsd.hostinit.mk file and put the build definitions for all host 2018-05-01 19:59:42 +00:00
disklabel Use ${_TOOL_PREFIX} instead of hardcoding "nb" for consistency. 2019-05-17 13:51:36 +00:00
dtc Create a new bsd.hostinit.mk file and put the build definitions for all host 2018-05-01 19:59:42 +00:00
elftoolchain Bug fix: use the correct target path in a dependency. 2022-05-02 18:28:38 +00:00
elftosb remove trailing newline 2018-05-19 17:48:51 +00:00
fdisk Patch the build script and toolchain to allow passing through 2021-09-18 01:47:07 +00:00
fgen
file force disable seccomp, in case it gets detected. 2018-09-28 14:45:16 +00:00
gcc bump the netbsd version GCC to yesterday's date. 2022-07-23 19:01:18 +00:00
gdb Set ac_cv_func_sigprocmask=yes as the symbol renaming confuses the 2023-05-08 09:47:20 +00:00
genassym Create a new bsd.hostinit.mk file and put the build definitions for all host 2018-05-01 19:59:42 +00:00
gencat
gettext Create a new bsd.hostinit.mk file and put the build definitions for all host 2018-05-01 19:59:42 +00:00
gmake
gmp Patch the build script and toolchain to allow passing through 2021-09-18 01:47:07 +00:00
gpt Use ${_TOOL_PREFIX} instead of hardcoding "nb" for consistency. 2019-05-17 13:51:36 +00:00
grep Add WITHOUT_GZIP 2018-08-12 09:03:42 +00:00
groff
hexdump
host-mkdep tools: configure --silent if MAKEVERBOSE == 0 2020-06-13 10:49:17 +00:00
hp300-mkboot
hppa-mkboot
ibmnws-ncdcs
installboot Add evbmips support for u-boot handling. 2020-06-21 17:17:01 +00:00
join
lex PR/53544: Add a dependency on the Makefile to fix incremental builds. 2018-08-23 06:21:37 +00:00
libctf Prepare the ground for <elfdefinitions.h> to be used by the 'tools' 2022-05-03 10:50:42 +00:00
libfdt Create a new bsd.hostinit.mk file and put the build definitions for all host 2018-05-01 19:59:42 +00:00
libprop Fix building libprop as a host tool library on platforms that don't have 2019-05-08 02:25:50 +00:00
lint lint: make path to lint data configurable 2021-01-24 16:55:09 +00:00
lint1
lint2
llvm Update LLVM build system for 249b40b558955afe5ac2b549edcf2d7f859c8cc9 2021-05-30 01:56:44 +00:00
llvm-clang Update LLVM to 10.0.0git (01f3a59fb3e2542fce74c768718f594d0debd0da) 2019-11-11 22:44:56 +00:00
llvm-clang-tblgen Update LLVM to 10.0.0git (01f3a59fb3e2542fce74c768718f594d0debd0da) 2019-11-11 22:44:56 +00:00
llvm-include
llvm-lib Fix LLVM build with host gcc 13 2023-05-30 19:05:04 +00:00
llvm-tblgen Update LLVM to 10.0.0git (01f3a59fb3e2542fce74c768718f594d0debd0da) 2019-11-11 22:44:56 +00:00
lorder Create a new bsd.hostinit.mk file and put the build definitions for all host 2018-05-01 19:59:42 +00:00
m4
m68k-elf2aout s/hidding/hiding/ 2022-05-28 22:00:55 +00:00
macppc-fixcoff
make make(1): remove references to the removed lst.lib 2020-07-26 09:17:24 +00:00
makefs
makekeys Centralize NOMAN handling. 2018-05-01 12:38:39 +00:00
makestrs
makewhatis
mandoc Expose re{c,}allocarray decls in current. 2018-08-15 14:31:22 +00:00
mdsetimage Create a new bsd.hostinit.mk file and put the build definitions for all host 2018-05-01 19:59:42 +00:00
menuc
mips-elf2ecoff
mkcsmapper
mkdep
mkesdb
mklocale
mknod
mktemp
mkubootimage
mpc
mpfr
msgc
mtree
mvme68k-wrtvid PR port-mvme68k/56146 2021-05-06 13:23:36 +00:00
nbperf
pax
paxctl
pcc
pigz
pkg_install don't try to include the library if we are building it. 2020-01-17 16:50:51 +00:00
powerpc-mkbootimage
pwd_mkdb
rpcgen Update tools/compat/configure for new path of "rpc/types.h". 2019-06-22 13:42:53 +00:00
sed
sgivol
slc
sortinfo
sparkcrc
stat
strfile
sunlabel Patch the build script and toolchain to allow passing through 2021-09-18 01:47:07 +00:00
texinfo
tic tools: Build tic with full fat terminfo 2020-04-05 12:31:39 +00:00
tsort
uudecode Patch the build script and toolchain to allow passing through 2021-09-18 01:47:07 +00:00
veriexecgen
vgrind Patch the build script and toolchain to allow passing through 2021-09-18 01:47:07 +00:00
xz-bin Prepend ${_TOOL_PREFIX} to xz, too. 2018-11-19 20:00:28 +00:00
xz-include tools: use NONLS= not MKNLS=no 2023-06-02 20:47:27 +00:00
xz-lib Update build glue for xz-5.2.4. Hook up tools version. 2018-09-25 05:42:08 +00:00
yacc
zic don't require gettext 2022-10-29 13:53:35 +00:00
Makefile Disconnect 'tools/lib{elf,dwarf}' from the build. 2022-04-18 19:46:35 +00:00
Makefile.gmakehost
Makefile.gnuhost Fix passing -j NNN to gmake 2023-05-19 10:42:34 +00:00
Makefile.gnuwrap
Makefile.host tools: revert use of <bsd.hostinit.mk> 2023-06-04 16:53:59 +00:00
Makefile.nbincludes tools: use <bsd.hostinit.mk> where appropriate 2023-06-04 10:00:20 +00:00
README tools: allow tools to use C99 2022-02-03 20:32:38 +00:00
headerlist Add initial support for ARMv8 (AARCH64) (by nisimura@ and ryo@) 2018-04-01 04:35:01 +00:00
mkheaderlist.sh

README

$NetBSD: README,v 1.5 2022/02/03 20:32:38 rillig Exp $

Notes for NetBSD src/tools


Background
==========

Several programs that are part of NetBSD are also built as tools.  Such
programs are typically built twice: once as a tool and once as part of
the release build.  Tools are relevant only when the make(1) variable
USETOOLS=yes, which is the default for most NetBSD builds.

Tools are built on the host platform, using the host compiler,
and will run on the host platform during the cross-build of the
remainder of NetBSD.  They are built near the beginning of a NetBSD
build (e.g. "build.sh tools" or "make tools" from the top level src
directory), and installed in ${TOOLDIR}.

Tools are executed during the main part of the build, when several
TOOL_* variables defined in src/share/mk/bsd.*.mk will refer to the
tools installed in ${TOOLDIR}.


Portability
===========

Programs that are built as tools need to be more portable than other
parts of NetBSD, because they will need to run on the host platform.

Most tools should restrict themselves to C language features that are
defined in C99 (ISO/IEC 9899-1999); they should avoid using C11 language
features, such as <threads.h>, _Alignof, <uchar.h>, _Generic,
static_assert, anonymous structures and unions.

Tools may use library features such as functions, macros, and types,
that are defined in C99 and in POSIX (IEEE Std 1003.1) (XXX year?), and
features that are provided by the compatibility framework
(src/tools/compat) described in a separate section below.

If a tool attempts to use a feature that is not available on the host
platform, then the tools build will fail.  This can be addressed by
changing the tool to avoid that feature, or by adding the feature to the
src/tools/compat framework.  It is usually easy to add new macros or
functions to src/tools/compat, and that is usually better than adding
compatibility definitions to individual tools.


Compatibility framework
=======================

src/tools/compat provides a compatibility framework for use by tools.
It installs the following components, and more:

${TOOLDIR}/lib/libnbcompat.a

    A library containing functions that are needed by some tools.

${TOOLDIR}/include/nbtool_compat.h

    A header file defining macros that are needed by some tools.

${TOOLDIR}/share/compat/defs.mk

    A makefile fragment, to be included by other makefiles,
    to define make variables appropriate for building tools.

    Among other things, this makefile fragment automatically adds
    the libnbcompat.a library to the LDADD and DPADD variables,
    so that tools will be linked with that library, and adds
    -I${NETBSDSRCDIR}/tools/compat and -DHAVE_NBTOOL_CONFIG_H=1 to the
    HOST_CPPFLAGS variable, so that compiled programs can detect when
    they are being built as tools.


Adapting Makefiles for use with tools
=====================================

Makefiles under src/tools/*/Makefile should define the HOSTPROG
variable.  This is typically done by tools/Makefile.hostprog,
which is directly or indirectly included by all Makefiles in
src/tools/*/Makefile.

Makefiles in the non-tools part of the src tree can test whether or not
the HOSTPROG variable is defined, in order tell the difference between
building a tool and building part of a NetBSD release, and they may
alter their behavior accordingly.

For example, the Makefile may conditionally refrain from compiling and
linking certain files, and the Makefile may conditionally pass macros to
the compiler via constructs like this:

    .if defined(HOSTPROG)
    CPPFLAGS+= -DWITH_FEATURE_X=0 # exclude feature X from tools build
    .else
    CPPFLAGS+= -DWITH_FEATURE_X=1 # include feature X in release build
    .endif

Adapting Programs for use with tools
====================================

When a tool is being built, the C compiler should automatically be
invoked with -DHAVE_NBTOOL_CONFIG_H=1.  This is done as a result of
settings in ${TOOLDIR}/share/compat/defs.mk, which should be included
from src/tools/Makefile.host, which should be included directly or
indirectly from src/tools/*/Makefile.

A C source file can test whether the HAVE_NBTOOL_CONFIG_H macro is
defined, in order to tell whether or not it is being compiled as part of
a tool.

In order to obtain the definitions provided by the tools compatibility
framework, almost every C source file that is built as part of a tool
should have lines like these as the first non-comment lines:

    #if HAVE_NBTOOL_CONFIG_H
    #include "nbtool_config.h"
    #endif

To omit features from the tools version of a program, the program
may test the HAVE_NBTOOL_CONFIG_H macro, like this:

    #if HAVE_NBTOOL_CONFIG_H
       ... code to be used when built as a tool
    #else
       ... code to be used when built as part of a release
    #endif

It is often preferable to use macros whose names refer to the features
that should be included or omitted.  See the section on "Adapting
Makefiles for use with tools" for an example in which the Makefile
passes -DWITH_FEATURE_X=0 or -DWITH_FEATURE_X=1 to the compiler
according to whether or not the program is being built as a tool.  Then
the program can use code like this:

    #if WITH_FEATURE_X 
       ... code to be used when FEATURE X is desired,
       ... e.g. when being built as part of a release.
    #else
       ... code to be used when FEATURE X is not desired,
       ... e.g. when being built as a tool.
    #endif