299 lines
14 KiB
Plaintext
299 lines
14 KiB
Plaintext
Proposal: NetBSD System Installation Packages
|
|
=============================================
|
|
|
|
CONTENTS
|
|
--------
|
|
0. Introduction
|
|
1. System Packages
|
|
1.1 Package Format
|
|
1.2 Package Granularity
|
|
1.2.1 Root/User/Share separation
|
|
2. Package Sets
|
|
2.1 Set format
|
|
3. Creation of Packages and Sets
|
|
4. Modifications to the NetBSD installation process
|
|
A. Working Plan
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
0. Introduction
|
|
|
|
The current NetBSD installation process involves the downloading
|
|
of binary `sets', which the user can choose among at install time.
|
|
A set is a tarred, gzipped set of files, to be untarred relative
|
|
to '/'. No facility exists to choose convenient subsets of the files
|
|
in a set to be installed, or to remove a set which has been installed.
|
|
|
|
The current granularity of sets is very large, being divided into:
|
|
|
|
base -- general system binaries
|
|
comp -- compilers and related tools
|
|
etc -- system configuration files
|
|
games -- games and other amusements
|
|
man -- system manual pages
|
|
misc -- items not falling into other categories
|
|
secr -- items not exportable under US law
|
|
text -- text processing tools
|
|
xbase -- general X11R6 binaries
|
|
xcomp -- X11R6 development items
|
|
xfont -- X11R6 fonts
|
|
xserver -- X11R6 servers for various video hardware
|
|
|
|
Users who wish to install part of a set need to either install
|
|
the full set and then determine which files they need to remove,
|
|
or abandon the normal install process, and figure out which files
|
|
to unpack by hand. Similarly, if a set is later determined to
|
|
be unnecessary, the only way to remove it is to figure out which
|
|
files on the system belonged to that set, and remove them by hand.
|
|
|
|
When it comes time to upgrade a system which has been installed this
|
|
way, the usual procedure is to unpack a new version of each installed
|
|
set over the previous version. When a file is moved, renamed, or
|
|
removed in a newer version of a set, the old version often remains on
|
|
the system for some time. In at least one recent instance (the move
|
|
of /sbin/mountd to /usr/sbin/mountd) this has resulted in much
|
|
confusion, and large amounts of traffic on the relevant mailing lists.
|
|
|
|
The remainder of this document describes a proposed method of handling
|
|
these and other problems with the current install set system by
|
|
moving to the use of fine-grained `system packages', based on the
|
|
currently existing package system for third-party software, and
|
|
allowing users to choose among either `package sets' at the same
|
|
granularity as our current install sets, or individual `packages'
|
|
at a much finer level of granularity. In either case, the new system
|
|
would also greatly simplify upgrading or removal of such packages
|
|
and sets at a later time, and would allow tracking of dependencies
|
|
between the various sets and packages distributed as part of NetBSD.
|
|
|
|
First, the format of system packages in the proposed system is
|
|
discussed, followed by the format of package sets, which will serve
|
|
as a replacement for the current install sets. The creation of
|
|
packages in an automated fashion from a NetBSD source tree is
|
|
discussed as is the effect of this system on the NetBSD installation
|
|
process. An appendix discusses my work plan to implement this new
|
|
system.
|
|
|
|
It is hoped that this document will serve as a basis for discussion
|
|
of what is involved in changing NetBSD to use system packages for
|
|
system installation and upgrades, and that after several iterations
|
|
of discussion and revision, it will serve as a plan for the actual
|
|
implementation of this system.
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
1. System Packages
|
|
|
|
System packages will be the basic building blocks of a NetBSD system.
|
|
At install time, the user will choose which system packages to install,
|
|
subject to dependencies between packages. After system install,
|
|
users will be able to install additional packages or remove installed
|
|
packages. When it comes time to upgrade the system, packages can
|
|
be removed and reinstalled in a reliable fashion. All of this
|
|
functionality is already available for third-party software via the
|
|
use of the software package system in /usr/pkgsrc. This proposal
|
|
extends that functionality to the NetBSD system itself.
|
|
|
|
1.1 Package Format
|
|
|
|
System packages will be identical in format to the binary packages
|
|
used by the current third-party package system. This will allow the
|
|
same tools to be used for working with system packages as are
|
|
currently used for working with third-party packages. This will also
|
|
also allow the system to benefit from the fact that the workings of
|
|
the current package system are well understood.
|
|
|
|
1.2 Package Granularity
|
|
|
|
System packages will be at the granularity of groups of related tools
|
|
and their support files. Thus, `Kerberos', `UUCP', `Text formatting'
|
|
and `amd' might each be packages which depended on nothing but a few
|
|
base packages, while `C Development' and `Fortran development' might
|
|
be separate packages which each depended upon `Binutils' and `Base
|
|
EGCS utilities' packages. Packages sets, described below, would add
|
|
the ability to choose entire broad categories of software to install,
|
|
like todays install sets, while maintaining the ability to remove
|
|
individual packages later.
|
|
|
|
1.2.1 Root/User/Share separation
|
|
|
|
In order to support a variety of system configurations, it is crucial
|
|
that the new package system support the possibility of some part of
|
|
a system residing on a server and possibly being shared between
|
|
multiple machines on a network. A machine which has some filesystems
|
|
local and some shared must, at the very least, be able to add and
|
|
remove packages from local filesystems, and should be able to
|
|
determine what packages have been added or removed from the volumes
|
|
mounted over the network.
|
|
|
|
The most common shared configurations are to have a system share
|
|
/usr/share from the network, and have all other filesystems local,
|
|
or to share the entirety of /usr from the network, and maintain
|
|
local root and /var hierarchies, possibly as a single filesystem.
|
|
Other commonly shared hierarchies include /usr/X11R6 and /usr/pkg.
|
|
|
|
Two steps are necessary to support this type of sharing: the system
|
|
must be able to check separate repositories for packages installed
|
|
on different filesystems, and packages must be designed so as to
|
|
allow a client to install only those parts of the system which reside
|
|
on local filesystems.
|
|
|
|
The first of these is addressed by a set of patches described by
|
|
Alistair Crooks in a post to the netbsd-current mailing list on
|
|
Friday, September 18, 1998. These patches, which have not yet been
|
|
committed cause third-party software packages installed in /usr/pkg
|
|
to be registered in /usr/pkg/etc/pkg, and packages installed in
|
|
/usr/X11R6 to be registered in /usr/X11R6/etc/pkg. This could be
|
|
extended easily to allow sharing of system package installations by
|
|
having the new system X11R6 packages also use /usr/X11R6/etc/pkg
|
|
for package registration, to have system packages installed in /usr
|
|
use /usr/etc/pkg for package registration, and to have system
|
|
packages installed in / and /var use /etc/pkg for package
|
|
registration. This would allow all of the types of filesystem
|
|
sharing described above, without introducing too much complication
|
|
into the package system.
|
|
|
|
The second step, that of insuring that a client can choose to install
|
|
only the parts of the system which reside on local volumes can be
|
|
most easily addressed by careful consideration of package contents.
|
|
A look through the contents of the current install sets suggests
|
|
that relatively few packages will in fact need to install in more
|
|
than one of /, /usr, /usr/share and /usr/X11R6. Were such packages
|
|
split into separate components, based on filesystem boundaries,
|
|
users would easily be able to install only the parts which are local
|
|
in their particular configuration.
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
2. Package Sets
|
|
|
|
In moving to fine-grained system packages, it is important that
|
|
beginning users still be able to select broad categories of software
|
|
to install at once. The introduction of `package sets', analogous
|
|
in granularity, but not mechanism, to the current binary install sets
|
|
addresses this concern, while maintaining the ability of more advanced
|
|
users to choose among individual packages at install time, and
|
|
maintaining the ability to remove, upgrade, or add individual
|
|
packages at a later time.
|
|
|
|
These package sets will maintain the same layout as the current
|
|
install sets, so that a user who chooses the same sets as he would
|
|
have chosen now will see the same results. In the new system,
|
|
however, these sets will be made up of binary packages, and installing
|
|
a set will simply result in the installation of the constituent
|
|
packages.
|
|
|
|
2.1 Set format
|
|
|
|
A set will be a tar archive containing the packages which make up the
|
|
set plus a contents file. At the least, the index file will contain
|
|
the name of each included package, plus a one line description of each
|
|
package's contents. Installation utilities will offer the option of
|
|
installing the whole set, or choosing among individual packages,
|
|
based on the descriptions in the contents file. It is expected that the
|
|
contents file itself will be automatically generated from the one-line
|
|
descriptions provided in each package's pkg/COMMENT file.
|
|
|
|
When a set is installed, the contents file will be recorded in a
|
|
manner similar to the registration of package information in the
|
|
current third-party package system. This will allow users to remove
|
|
an entire set at a later date, without needing to know what individual
|
|
packages came from that set.
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
3. Creation of Packages and Sets
|
|
|
|
Under the current distribution-building system, the Makefile in
|
|
/usr/src/etc creates binary install sets from an installed system,
|
|
based on the set lists in /usr/src/distrib/sets/lists. In the new
|
|
system, a new directory hierarchy, /usr/src/distrib/pkg, will
|
|
contain Makefiles and data files relevant to the creation of
|
|
system packages and package sets.
|
|
|
|
The directory /usr/src/distrib/pkg/sets will contain a directory
|
|
for each package set, and each of these directories will contain
|
|
a directory for each package in that set. The Makefile in
|
|
/usr/src/distrib/pkg/sets will recurse into these set directories
|
|
to build each set. The individual set Makefiles will recurse into
|
|
each package directory to build the individual packages, and will
|
|
then create a set file from the constituent packages and from the
|
|
contents file, which will be automatically generated from the
|
|
package directories.
|
|
|
|
The package directories will resemble the package directories for
|
|
third-party software packages in /usr/pkgsrc, except that they will
|
|
probably rely on the files making up the package already being
|
|
present in ${DESTDIR}, rather than building them directly. This
|
|
assumption is already present in the current distribution package
|
|
Makefile code, and is probably reasonable to keep.
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
4. Modifications to the NetBSD installation process
|
|
|
|
Once the NetBSD system is available as system packages and package
|
|
sets, it will be possible to modify the various installation tools
|
|
to use these sets to install the system. It is expected that
|
|
installation tools will default to allow users to choose among
|
|
package sets at install time, but allow an `advanced mode' in which
|
|
packages could be selected and deselected on an individual basis.
|
|
|
|
This will require that the various package tools (at least pkg_add)
|
|
be present on install media to be used with system packages.
|
|
Modifications to sysinst and other install tools are beyond the
|
|
current scope of this proposal, but will be necessary to take
|
|
advantage of the new capabilities provided by this system.
|
|
|
|
------------------------------------------------------------------------
|
|
|
|
A. Working Plan
|
|
|
|
My current plan for implementing system packages and package sets
|
|
for NetBSD consists of four steps. All of these steps should be
|
|
taken in the CVS source tree (segregated into src/distrib/pkg, of
|
|
course), and hopefully will involve other contributors in addition
|
|
to myself:
|
|
|
|
1.) Hammer this proposal into a more detailed specification
|
|
|
|
I am submitting this proposal now in the hopes that it
|
|
will spark discussion which will lead to a refinement
|
|
of the planned system package system. Once some sort
|
|
of consensus is reached on the relevant mailing lists,
|
|
I will begin work in earnest on implementing this.
|
|
|
|
2.) Create the /usr/src/distrib/pkg hierarchy, and a template
|
|
package
|
|
|
|
The first step in actually implementing this system will
|
|
be to create either an actual or mocked-up system package
|
|
which can be used as a template for creation of the
|
|
remaining system packages.
|
|
|
|
3.) Create system packages
|
|
|
|
I expect that this step will involve most of the actual
|
|
work in implementing the new system. Packages will have
|
|
to be created for each functional group of binaries
|
|
currently shipped with NetBSD. A lot of discussion and
|
|
design will have to go into the decisions as to how
|
|
many packages should make up each set and what files
|
|
belong in which packages.
|
|
|
|
4.) Create Package Sets
|
|
|
|
Once all system packages exist, it will be necessary to
|
|
put together some code to automatically generate set
|
|
contents files and to create sets from each directory
|
|
of packages in /usr/src/distrib/pkg/sets.
|
|
|
|
Once these steps are complete, NetBSD will have system packages,
|
|
and it will be possible to begin looking at modifying the NetBSD
|
|
install process to use them. It is important to note that none
|
|
of these changes will require modifying the current installation
|
|
set building code in any way, so the use of the current system
|
|
can continue unhindered while the new system is being implemented.
|
|
|
|
------------------------------------------------------------------------
|
|
$Id: PROPOSAL,v 1.2 2004/01/17 05:30:01 lukem Exp $
|