NetBSD/distrib/syspkg/notes/NOTES

I have been spending a lot of time lately thinking about how we
can extend our current package system to allow the use of system
packages (and sets thereof) rather than tarballs as our primary
way of installing NetBSD systems.  What follows is a random jumble
of thoughts on where we can go with this.  Based on people's response
to this, I will put together a more concrete proposal soon.

1. Package Granularity

    People would like to be able to install/remove system software
    with a high level of granularity.  (Do I want Kerberos, but
    not UUCP?  C but not Fortran?  Text formatting but not printing?)
    This could be best addressed by providing our system software
    as a series of packages which could be installed individually.
    Dependencies would then exist among these packages, which would
    insure that a working system resulted from various combinations
    users might choose.  For example, users could choose a C package
    and/or a Fortran package, and both would depend on a binutils
    package.

    At the same time, users should not need to choose between a
    thousand little packages when installing the system.

    This probably means that there should be the concept of `sets'
    of packages to be installed all at once, roughly analogous to
    our current breakdown into base, comp, etc, games and so on.
    A set would contain a number of packages, and a contents file
    indicating which packages were part of the set.  An install
    program could then default to let users pick among whole sets,
    but could have an advanced mode which would let users pick
    individual packages instead.

    Very possibly, this would open the possibility of keeping set,
    information, in addition to package information around, and
    offering the ability to remove all installed packages from a
    given set at a later date.

    Alternately, a system similar to our existing pkg-src/meta-pkgs
    mechanism could be used -- users would choose meta-pkgs more
    or less equivalent to the current sets, and these would trigger
    dependencies on the packages which make up the set.

2. Package Registration

    Currently, packages are installed in /usr/pkg and /usr/X11R6,
    and all package registration is done by creating directories
    in /var/db/pkg.

    This is generally agreed to have a couple of problems, especially
    when you want to share /usr/pkg between multiple machines.
    Unless /var/db/pkg is the same on both machines, you very
    quickly end up with at least one machine not knowing what
    packages it has installed.

    Alistair Crooks has recently put together changes to use
    ${LOCALBASE}/etc/pkg and ${X11BASE}/etc/pkg to store package
    registration information, which is a big step toward solving
    this problem (yahoo!).  Presumably, system packages will use
    /etc/pkg, although this leaves open the question of what to do
    if you share /usr between multiple client machines.

3. Package Creation

    Right now, the Makefile in .../src/etc can be used to build
    distribution sets, which are tarballs to be unpacked relative
    to / by the installer.  Sets are defined by lists in
    .../src/distrib/sets/lists, and consist of MI and MD components.
    Thus, where sources are installed in /usr/src,
    /usr/src/distrib/sets/lists/comp/mi contains a list of files
    in the compiler set which are available on all architectures,
    while /usr/src/distrib/sets/lists/comp/md.i386 contains a list
    of those files which are on the i386, but not on all other
    architectures.  All shared libraries, for example, are in the
    md set, since not all ports have shared libraries.

    In order to move to using system packages, we will need to
    provide a way to generate binary packages from our source tree.
    It should be noted that the granularity of packages does not
    really match the granularity of source subdirectories in
    /usr/src/*, so barring a source tree re-organization (which
    seems undesirable), we will need a way to do this which (like
    the current mechanism) is outside of the normal `make clean ;
    make depend; make' recursion through the source tree.  A system
    will also be needed to allow building a set out of a group of
    packages.  Both of these systems will need to be simple to use
    (one make command line), and should be possible to run on a
    single-package or single-set level.

4. Package Versioning and Upgrades

    One of the neatest features offered by having independent
    packages for different parts of the system is the ability to
    upgrade a single package independently of the whole system.
    Although this should not be abused (we don't want to become
    like Linux where on an average system nothing quite matches
    anything else), it would provide an ideal way to distribute
    important patches and security fixes.

    For example, the security patch for NetBSD-SA1998-004 (at(1)
    being usable to view any file) was distributed as a patch to
    the NetBSD-1.3.2 source tree.  This requires anyone who wishes
    to install the fix to have a source tree available, download
    and apply the patch, and then rebuild at(1).

    In a system based on install systems, an updated version of
    the specific package containing at(1) could be released, which
    users could download and pkg_add in a matter of moments.  It
    would also be easy for an admin to tell if a system had been
    upgraded by looking at pkg_info output.

$Id: NOTES,v 1.1.1.1 2002/01/07 22:46:16 jwise Exp $