$NetBSD: UPDATING,v 1.98 2003/11/20 08:07:52 wiz Exp $ This file is intended to be a brief introduction to the build process and a reference on what to do if something doesn't work. For a more detailed description see Makefile. Recent changes: ^^^^^^^^^^^^^^^ 20031111: A newer mkdep is needed. Error noting that is cc: Ambiguous abbreviation -- 20031008: /usr/include/sys/disklabel_mbr.h was removed. It's necessary to "make cleandir" to ensure that the dependencies will be rebuilt correctly. 20030906: With the addition of siginfo support the old signal trampoline code has been deprecated to COMPAT_16. Make sure that your running kernel has COMPAT_16 enabled before building userland. 20030801: With the new openssl, there is some header and library shuffling. rm -f /usr/include/des.h /usr/include/kerberosIV/* /lib/libdes* \ /usr/lib/libdes* before building. 20030703: Texinfo was updated to 4.6. To avoid failures when trying to build the included texinfo files, do: cd src/gnu/usr.bin/texinfo make MKINFO=no dependall install 20030630: Groff was update to 1.19; it's probably necessary to do cd share/mk && make install cd src/gnu/usr.bin/groff make MKMAN=no dependall install (untested). 20030516: Due to bugs in the export handling code, invalid export lines were accepted before and caused the kernel to panic when mountd got restarted because it freed memory that had already been freed. This has been fixed and the kernel checks export addresses very strictly. If you upgrade your kernel, make sure you also upgrade mountd, because if your export file contains lines with an old inet4 address syntax (i.e. a.b.c or a.b or a), they will get rejected by the new kernel. 20030402: The superblock layout for FFS was changed. If you have 1.6 fsck binaries, they will signal a fatal superblock mismatch with the first alternate, because they compare too many fields (even ones that aren't useful). If possible, upgrade your fsck_ffs binary before using a new kernel. None of this signals actual filesystem damage. 20030324: sendmail version 8.12.8 was imported. Since sendmail is now setgid to the smmsp group, and runs in "collection" mode for most common activities, there is a new config file called submit.cf that needs to live in /etc/mail. The generic submit.cf sample in /usr/share/sendmail/cf is named netbsd-msp.cf. Upgrading your regular sendmail configuration file is also strongly advised. See the section named "MESSAGE SUBMISSION PROGRAM" in the updated /usr/share/sendmail/README file for more information. 20030117: Texinfo was updated to 4.3. To avoid failures when trying to build the included texinfo files, do: cd src/gnu/usr.bin/texinfo make MKINFO=no dependall install 20021223: The METALOG format changed slightly, to remove the leading "${DESTDIR}" from path names. This only affects people building with UNPRIVED. For complete safety, remove the DESTDIR entirely and update tools/mtree, before running make build. 20021219: CVS repository layout was changed. See the following for details if you are using (anonymous) cvs to update your tree. http://mail-index.NetBSD.org/netbsd-announce/2002/12/19/0000.html 20021219: install(1) had a '-N dbdir' option added, to specify an alternate location to look up users & groups (instead of the host system passwd(5) and group(5) databases). The build system was modified to take advantage of this option (using ${NETBSDSRCDIR}/etc), so if you use USETOOLS==no, you may have to rebuild and reinstall usr.bin/xinstall first. 20021130: fparseln(3) moved from libutil to libc. If building to DESTDIR=/, reinstall the includes and rebuild libc: make includes make do-lib-libc If using build.sh, "cd tools/compat && make clean" before rebuilding the tools. 20021126: The mk.conf(5) variable SYS_INCLUDE has been deprecated, including the optional "SYS_INCLUDE=symlinks" support. All header files, including are copied into /usr/include. 20021121: The C run-time support files crtbegin.o and crtend.o (and their companions crtbeginS.o and crtendS.o) were split up, with new crti.o and crtn.o files resulting. This means that libtool needs to be rebuilt once the new libraries are installed. The process of rebuilding libtool will cause it to automatically notice the new required files, but it *must* be rebuilt in order to do this. An out-of-date libtool will result in shared libraries which lack _init() and _fini() routines, which means that their global contructors/destructors will not be invoked. 20021121: A bug related to how ARM ELF objects were tagged has been corrected. NetBSD ARM ELF uses the soft-VFP floating point model by default. However, the assembler lacked support for marking objects as using the VFP floating point format, and the compiler was not properly passing the flag indicating "soft-VFP" to the assembler. Unfortunately, this means that the linker will now consider old (i.e. not marked "softvfp") NetBSD ARM ELF objects to be incompatible with new (properly marked) objects. The problem will only manifest itself if you attempt to compile a new program using the fixed toolchain, and link that program against old libraries which do not have the proper "softvfp" markings. ALL OF YOUR EXISTING BINARIES AND SHARED LIBRARIES WILL CONTINUE TO WORK PROPERLY. The only work-around for the problem is to recompile all of the libraries on the system. The easiest way to do this for system libraries is to install a binary snapshot; they are generally available on releng.NetBSD.org. Any packages you have installed which supply libraries will have to be recompiled if you wish to link new programs against those libraries. If you have questions about this matter, please contact port-arm@NetBSD.org. 20021011: Systrace has been improved to support privilege elevation. Updating the kernel requires the userland part of systrace to be rebuilt. 20021010: The config(8) grammar was changed to allow options to register dependencies on attributes, as well as other options. Users must update and reinstall usr.sbin/config before building a new kernel. 20021009: A new attribute dependency syntax was introduced to config(8), which is now used by the SCSI configuration description. Users must update and reinstall usr.sbin/config before building a new kernel. 20021003: Several changes have been made to the autoconfiguration framework. Users must update and reinstall usr.sbin/config before building a new kernel. 20021001: The i386mp branch has been merged. To compile a kernel, users will need to add the option 'cpu* at mainbus?' to their configuration file. Multiprocessor kernels will need ioapic* at mainbus? apid ? options MULTIPROCESSOR options COM_MPLOCK 20020922: MKDYNAMICROOT=yes enabled by default, which means that certain shared libraries are installed into /lib, the shared linker is installed into /libexec, and all programs in /bin and /sbin are dynamically linked. If you do not use "make build", you should ensure that you have the libraries and shared linker in the new locations, with: make do-lib-csu do-lib-libc do-lib do-gnu-lib do-ld.elf_so 20020917: USE_NEW_TOOLCHAIN has been replaced with: - TOOLCHAIN_MISSING -- set to "yes" on platforms for which there is no working in-tree toolchain (hppa, ns32k, sh5, x86_64). - EXTERNAL_TOOLCHAIN -- if defined by the user, points to the root of an external toolchain (e.g. /usr/local/gnu). This enables the cross-build framework even for TOOLCHAIN_MISSING platforms. 20020906: gehenna-devsw has been merged into the trunk. Need to update and reinstall usr.sbin/config before building the kernel. 20020822: Crunched rescue tools (contents of /bin and /sbin, plus others) are now provided in /rescue. To ensure that these are built statically linked (no matter what the setting of LDSTATIC is), use a crunchgen(1) built from sources newer than 20020820 (see the next entry). 20020820: crunchgen(1) changed to ensure that the generated program is statically linked. Solution: update and reinstall usr.bin/crunch 20020605: smmsp user/group has been added for sendmail. Add the following into /etc/group: smmsp:*:17: and the following to /etc/master.passwd (via vipw): smmsp:*:17:17::0:0:Sendmail Message Submission Program:/nonexistent:/sbin/nologin 20020515: sshd user/group has been added. Need to hand add this in, or sshd will not let you log in (with default, or UsePrivlegeSeparation=yes) Add the following into /etc/group: sshd:*:16: and the following to /etc/master.passwd (via vipw): sshd:*:16:16::0:0:& pseudo-user:/var/chroot/sshd:/sbin/nologin Also /var/chroot/sshd directory needs to be present (digged as part of the build process). 20020426: NBUILDJOBS obsoleted in favor of just using -j. 20020426: etc/postinstall added, which performs various checks for configuration file updates and changes, and can fix most of the problems identified. This should make it much easier to upgrade a system's configuration from earlier systems (as far back as NetBSD 1.5). 20020320: needs a new install(1) for its "-a cmd" support. Build and install at usr.bin/xinstall before the build. 20020319: Raw IPv6 socket now makes strict checking for sa_family and sa_len on send(2) operation. Be sure to have sbin/rtsol and usr.sbin/rtsold newer than November 2001 when you upgrade the kernel. 20020311: ssh configuration files were moved from /etc to /etc/ssh. Beware if you restart your machine from remote. Note that sshd.conf needs to be changed (due to the use of "/etc" inside). 20020223: Users of the VAX port will need to rebuild and install gas so it deal with the now present register prefix used in all the VAX assembly files. 20020118: ntpd user/group has been added. Need to hand add this in or builds will break as mtree aborts early. Add the following into /etc/group: ntpd:*:15: and the following to /etc/master.passwd (via vipw): ntpd:*:15:15::0:0:Ntpd pseudo-user:/var/chroot/ntpd:/sbin/nologin 20011207: If you're attempting to build a snapshot on sparc64 and are getting reloc errors from the toolchain groff binary this means your native toolchain has some broken C++ bits. To fix: Build a new toolchain (i.e. build.sh -t) Use the new toolchain to build and install natively (i.e. /usr/lib) gnu/lib/libgcc gnu/lib/libstdc++ After this a snapshot will be able to be built. 20011201: In order for a sparc64 build to work you must have a working awk. If you've built and installed a system with the new toolchain up to this point you do not have a working awk as its ability to do floating point is broken. To build: remake and install gnu/lib/libgcc remake and install gnu/usr.bin/gawk into /usr/bin (make sure it links against the new libgcc.a) 20011128: Kernel config information was changed to use defflag in the various "files" files. Bug fixes to config(8) are required in order for this to work properly. Make sure to build and install in usr.sbin/config before attempting to build a new kernel. 20011030: libc/locale/wcstod.c now needs new lint(1). Update lint(1) before building libc. 20011029: The new document BUILDING.mdoc (view with nroff | more, or see pre-generated .txt and .html versions) describes the build procedure in great detail. BUILDING, and the USE_NEW_TOOLCHAIN build process, are intended in the long run to replace this manual update log. Users building a USE_NEW_TOOLCHAIN system should read the BUILDING document for caveats. Generally, BUILDING supersedes UPDATING for these systems, as tool updating is taken care of by the new build system. 20011028: src/etc/Makefile now needs install to be able to handle symlinks that point to nowhere. A bug in install that prevented this was corrected. Solution: update and reinstall usr.bin/xinstall Better Solution: Use the new toolchain and it will just work for you. 20011006: /etc/mtree/NetBSD.dist has been updated to take advantage of absolute path support added to mtree(8). Older mtree(8)s don't understand the format. Solution: update and reinstall usr.sbin/mtree 20011004: Crunchgen has been updated to work via reach-over makefiles. Updating is suggested before running a snapshot build 20010915: The new "ubcperf" code committed by Chuck Silvers removed a header file, uvm/uvm_vnode.h. There may be stale .depend files that still reference this file. Solution: "make cleandir && make dependall" in affected directories. 20010803: grep.info is now built from grep.texi using makeinfo. Since it requires makeinfo v4.0, you need to install new texinfo before building gnu/usr.bin/grep. To install new texinfo, please follow the instruction described in 20010726 entry. 20010803: (i386 only): i386 kernel now uses new instructions like `fxsave' which old gas doesn't understand. To build the kernel successfully, you need to build and install a new toolchain, (i.e., build.sh -t) or (temporarily) comment out "options I686_CPU" from your kernel configuration until you rebuild your userland. See 20011029 above and BUILDING file in this directory for more information. [updated 20020630 since i386 gas moved when USE_NEW_TOOLCHAIN enabled] 20010731: Bootloader update on ELF platforms. DDB in kernels from before this will be unable to read symbol tables provided by newer bootloaders. 20010726: Texinfo was updated to 4.0. To avoid failures when trying to build the included texinfo files, do: cd src/gnu/usr.bin/texinfo make MKINFO=no dependall install 20010718: Enabled correct .init/.fini processing in crt0. The way this was done was to change a -I directive to cc(1), which means make(1) will have a stale dependency (it will be checking the timestamp on the wrong "dot_init.h"). The symptom you will see is that new programs die with SIGSEGV if you have a stale dependency. Solution: "make cleandir" in both lib/csu and libexec/ld.elf_so before starting your build. 20010628: A construct was added to uvm_page.h that uncovered a bug in lint(1). If you get a warning/error about a non-portable bitfield, update your lint(1) before proceeding. 20010226: Added named user/group to system. Need to hand add this in or builds will break as mtree aborts early. To work around add by hand: named:*:14: to /etc/group and add: named:*:14:14::0:0:Named pseudo-user:/var/named:/sbin/nologin to master.passwd (use vipw for instance if doing by hand). Now a make build should progress. 20010219: get/setprogname() added. Any hostprogs that may use this will need to be bootstrapped manually until the host system is current. Known problems: sys/arch/macppc/stand/fixcoff usr.sbin/config (adding -DMAKE_BOOTSTRAP to CFLAGS and rebuilding should work) usr.sbin/mdsetimage - Build a static copy if building a snapshot before fully bootstrapped. 20010204: prepare the code to compile with stricter gcc flags. in particular start eliminating redundant declarations. Yacc needs to be installed before make build. 20010114: introduce .if commands(target) in make(1). You need to bring everything up-to-date first, then without installing anything make and install in usr.bin/make, then proceed with make build. 20010101: bsd.subdir.mk committed 20001230 had a bug which caused afterinstall targets to run too soon; update again. 20001230: New share/mk files needed to support .WAIT in SUBDIR variables. If you get make errors, (cd share/mk; make install) Also, PRINTOBJDIR has changed and is now used more heavily. 20001019: The `ca' device driver has been replaced by `ld'; although the major and minor numbers haven't changed, you should update your /dev directory. 20000929: The following make directives are obsoleted. MKCRYPTO_RSA NOCRYPTO_RSA NOCRYPTO_RC5 NOCRYPTO_IDEA By default, RSA is built into libcrypto. IDEA and RC5 will not be built into libcrypto. By using MKCRYPTO_{RC5,IDEA}, you can build additional library libcrypto_{idea,rc5}. Hints for a more successful build: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Build a new kernel first: This makes sure that any new system calls or features expected by the new userland will be present. This helps to avoid critical errors when upgrading. Use object directories: This helps to keep stale object files from polluting the build if a Makefile "forgets" about one. It also makes it easier to clean up after a build. It's also necessary if you want to use the same source tree for multiple machines. To use object directories: a) cd /usr/src ; make cleandir b) Add "OBJMACHINE=yes" to /etc/mk.conf c) Add "MKOBJDIRS=yes" to /etc/mk.conf d) cd /usr/src ; make build Note that running "make obj" in a directory will create in obj.$MACHINE directory. Build to a DESTDIR: This helps to keep old installed files (especially libraries) from interfering with the new build. To build to a DESTDIR, set the DESTDIR environment variable before running make build. It should be set to the pathname of an initially empty directory. Problems: you might need to update critical utilities without using DESTDIR since nothing is executed from what is installed in DESTDIR. (See critical utils, below) Build often: This keeps critical utilities current enough to not choke on any other part of the source tree that depends on up to date functionality. What to do if things don't work: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When things don't work there is usually a few things that commonly should be done. 1) make includes This should be done automatically by make build. 2) cd share/mk && make install Again, automatically done by make build. Failsafe rebuild of a small part of the tree: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ To make sure you rebuild something correctly you want to do something like the following: 1) Make sure the includes and .mk files are up to date. 2) Make sure any program used to build the particular utility is up to date. (yacc, lex, etc...) 3) cd ...path/to/util... make cleandir rm ...all obj directories... make cleandir # yes, again make obj make depend && make Failsafe rebuild of the entire tree: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ If you really want to make sure the source tree is clean and ready for a build try the following. Note that sourcing /etc/mk.conf (a make(1) Makefile) in this manner is not right, and will not work for anyone who uses any make(1) features in /etc/mk.conf. ---cut here--- #!/bin/sh . /etc/mk.conf if [ -z $NETBSDSRCDIR ] ; then NETBSDSRCDIR=/usr/src fi if [ \! -d $NETBSDSRCDIR ] ; then echo Unable to find sources exit 1 fi find $NETBSDSRCDIR -name \*.o -o -name obj.\* -o -name obj -exec rm \{\} \; if [ -z $BSDOBJDIR ] ; then BSDOBJDIR=/usr/obj fi if [ -d $BSDOBJDIR ] ; then rm -rf $BSDOBJDIR fi cd $NETBSDSRCDIR && make cleandir ---cut here--- Critical utilities: ^^^^^^^^^^^^^^^^^^^ gnu/usr.bin/egcs usr.bin/compile_et usr.bin/make usr.bin/yacc usr.bin/lex usr.bin/xlint usr.sbin/config Other problems and possible solutions: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Symptom:Unreasonable compiler errors. Fix: Rebuild gnu/usr.bin/egcs Symptom:Complaints involving a Makefile. Fix: Rebuild usr.bin/make: cd usr.bin/make && make && make install Or, a failsafe method if that doesn't work: cd usr.bin/make && cc *.c */*.c -I . -o make && mv make /usr/bin Fix: Make sure .mk files are up to date. cd share/mk && make install Symptom:Kernel `config' fails to configure any kernel, including GENERIC. Fix: Rebuild usr.sbin/config Symptom: Fix: Rebuild usr.bin/yacc Symptom: Fix: Rebuild usr.bin/lex Symptom: Fix: rm /usr/lib/libbfd.a Symptom:Obsolete intermediate files are used during compilation Fix: Try the following sequence of commands in the directory in question. make cleandir; rm `make print-objdir`; make cleandir; make obj (If you built the tree without "make obj" in the past, obsolete files may remain. The command tries to clean everything up) Symptom:.../sysinst/run.c:xx: warning: initialization from incompatible pointer type Fix: Rebuild and install usr.bin/menuc Symptom:mklocale not found during build in share/locale/ctype Fix: Build and install usr.bin/mklocale Symptom:undefined reference to `__assert13' or `__unsetenv13' Fix: Rebuild and install lib/libc Symptom:usr.sbin/config fails to build. Fix: Try building with -DMAKE_BOOTSTRAP added to CFLAGS in Makefile. Symptom:undefined reference to `getprogname' or `setprogname' Fix: Rebuild and install lib/libc Symptom:lint does not understand the '-X' option Fix: May need to build & install libs with NOLINT=1 before rebuilding lint