   * Am-utils: (am-utils).          The Amd automounter suite of
utilities  direntry

   Am-utils - The 4.4 BSD Automounter Tool Suite
*********************************************

   Am-utils is the 4.4 BSD Automounter Tool Suite, which includes the
Amd automounter, the Amq query and control program, the Hlfsd daemon,
and other tools.  This Info file describes how to use and understand the
tools within Am-utils.

Indexes

License
*******

   Am-utils is not in the public domain; it is copyrighted and there are
restrictions on its distribution.

   Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are
met:

  1. Redistributions of source code must retain the above copyright
     notice, this list of conditions and the following disclaimer.

  2. Redistributions in binary form must reproduce the above copyright
     notice, this list of conditions and the following disclaimer in the
     documentation and/or other materials provided with the
     distribution.

  3. All advertising materials mentioning features or use of this
     software must display the following acknowledgment:

     "This product includes software developed by the University of
     California, Berkeley and its contributors, as well as the Trustees
     of Columbia University."

  4. Neither the name of the University nor the names of its
     contributors may be used to endorse or promote products derived
     from this software without specific prior written permission.

   THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS
BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
THE POSSIBILITY OF SUCH DAMAGE.

Source Distribution
*******************

   You can get the latest distribution version of Am-utils from
     ftp://ftp.cs.columbia.edu/pub/am-utils/am-utils.tar.gz

   Alpha and beta distributions are available in
     ftp://shekel.mcl.cs.columbia.edu/private/am-utils/.

   Revision 5.2 was part of the 4.3 BSD Reno distribution.

   Revision 5.3bsdnet, a late alpha version of 5.3, was part of the BSD
network version 2 distribution

   Revision 6.0 was made independently by Erez Zadok at Columbia
University, as part of his PhD thesis work. See History for more
details.

Bug Reports
===========

   Send all bug reports to `amd-dev@majordomo.cs.columbia.edu' quoting
the details of the release and your configuration.  These can be
obtained by running the command `amd -v'.  It would greatly help if you
could provide a reproducible procedure for detecting the bug you are
reporting.

Mailing List
============

   There are two mailing lists for people interested in keeping
up-to-date with developments.

  1. The older list, `amd-workers' is for general "how to" questions and
     announcements.  To subscribe, send a note to
     `amd-workers-request@majordomo.glue.umd.edu'.(1)  To post a
     message to this list, send mail to
     `amd-workers@majordomo.glue.umd.edu'.

  2. The developers only  list, `amd-dev' is for

        - announcements of alpha and beta releases of am-utils

        - reporting of bugs and patches

        - discussions of new features for am-utils

        - implementation and porting issues

     To subscribe, send a note to `amd-dev@majordomo.cs.columbia.edu'.
     To post a message to this list, send mail to
     `amd-dev@majordomo.cs.columbia.edu'.  To avoid as much spam as
     possible, only subscribers to this list may post to it.

     Subscribers of `amd-dev' are most suitable if they have the time
     and resources to test new and buggy versions of amd, on as many
     different platforms as possible.  They should also be prepared to
     learn and use the GNU Autoconf, Automake, and Libtool packages,
     and of course, be very familiar with the complex code in the
     am-utils package.  In other words, subscribers on this list should
     be able to contribute meaningfully to the development of amd.

   ---------- Footnotes ----------

   (1)  Note that the older address,
`amd-workers-request@acl.lanl.gov', is defunct.

Introduction
************

   An "automounter" maintains a cache of mounted filesystems.
Filesystems are mounted on demand when they are first referenced, and
unmounted after a period of inactivity.

   Amd may be used as a replacement for Sun's automounter.  The choice
of which filesystem to mount can be controlled dynamically with
"selectors".  Selectors allow decisions of the form "hostname is THIS,"
or "architecture is not THAT."  Selectors may be combined arbitrarily.
Amd also supports a variety of filesystem types, including NFS, UFS and
the novel "program" filesystem.  The combination of selectors and
multiple filesystem types allows identical configuration files to be
used on all machines thus reducing the administrative overhead.

   Amd ensures that it will not hang if a remote server goes down.
Moreover, Amd can determine when a remote server has become
inaccessible and then mount replacement filesystems as and when they
become available.

   Amd contains no proprietary source code and has been ported to
numerous flavors of Unix.

History
*******

   The Amd package has been without an official maintainer since 1992.
Several people have stepped in to maintain it unofficially.  Most
notable were the `upl' (Unofficial Patch Level) releases of Amd,
created by me (Erez Zadok), and available from
ftp://ftp.cs.columbia.edu/pub/amd/.  The last such unofficial release
was `upl102'.

   Through the process of patching and aging, it was becoming more and
more apparent that Amd was in much need of revitalizing.  Maintaining
Amd had become a difficult task.  I took it upon myself to cleanup the
code, so that it would be easier to port to new platforms, add new
features, keep up with the many new feature requests, and deal with the
never ending stream of bug reports.

   I have been working on such a release of Amd on and off since
January of 1996.  The new suite of tools is currently named "am-utils"
(AutoMounter Utilities), in line with GNU naming conventions, befitting
the contents of the package.  In October of 1996 I had received enough
offers to help me with this task that I decided to make a mailing list
for these group of people.  Around the same time, Amd had become a
necessary part of my PhD thesis work, resulting in more work performed
on am-utils.

   Am-utils version 6.0 was numbered with a major new release number to
distinguish it from the last official release of Amd (5.x).  Many new
features have been added such as a GNU `configure' system, NFS Version
3, Autofs support, a run-time configuration file (`amd.conf'), many new
ports, more scripts and programs, as well as numerous bug fixes.
Another reason for the new major release number was to alert users of
am-utils that user-visible interfaces may have changed.  In order to
make Amd work well for the next 10 years, and be easier to maintain, it
was necessary to remove old or unused features, change various syntax
files, etc.  However, great care was taken to ensure the maximum
possible backwards compatibility.

Overview
********

   Amd maintains a cache of mounted filesystems.  Filesystems are
"demand-mounted" when they are first referenced, and unmounted after a
period of inactivity.  Amd may be used as a replacement for Sun's
automount(8) program.  It contains no proprietary source code and has
been ported to numerous flavors of Unix.  See Supported Platforms.

   Amd was designed as the basis for experimenting with filesystem
layout and management.  Although Amd has many direct applications it is
loaded with additional features which have little practical use.  At
some point the infrequently used components may be removed to streamline
the production system.

Fundamentals
============

   The fundamental concept behind Amd is the ability to separate the
name used to refer to a file from the name used to refer to its physical
storage location.  This allows the same files to be accessed with the
same name regardless of where in the network the name is used.  This is
very different from placing `/n/hostname' in front of the pathname
since that includes location dependent information which may change if
files are moved to another machine.

   By placing the required mappings in a centrally administered
database, filesystems can be re-organized without requiring changes to
configuration files, shell scripts and so on.

Filesystems and Volumes
=======================

   Amd views the world as a set of fileservers, each containing one or
more filesystems where each filesystem contains one or more "volumes".
Here the term "volume" is used to refer to a coherent set of files such
as a user's home directory or a TeX distribution.

   In order to access the contents of a volume, Amd must be told in
which filesystem the volume resides and which host owns the filesystem.
By default the host is assumed to be local and the volume is assumed to
be the entire filesystem.  If a filesystem contains more than one
volume, then a "sublink" is used to refer to the sub-directory within
the filesystem where the volume can be found.

Volume Naming
=============

   Volume names are defined to be unique across the entire network.  A
volume name is the pathname to the volume's root as known by the users
of that volume.  Since this name uniquely identifies the volume
contents, all volumes can be named and accessed from each host, subject
to administrative controls.

   Volumes may be replicated or duplicated.  Replicated volumes contain
identical copies of the same data and reside at two or more locations in
the network.  Each of the replicated volumes can be used
interchangeably.  Duplicated volumes each have the same name but contain
different, though functionally identical, data.  For example,
`/vol/tex' might be the name of a TeX distribution which varied for
each machine architecture.

   Amd provides facilities to take advantage of both replicated and
duplicated volumes.  Configuration options allow a single set of
configuration data to be shared across an entire network by taking
advantage of replicated and duplicated volumes.

   Amd can take advantage of replacement volumes by mounting them as
required should an active fileserver become unavailable.

Volume Binding
==============

   Unix implements a namespace of hierarchically mounted filesystems.
Two forms of binding between names and files are provided.  A "hard
link" completes the binding when the name is added to the filesystem.  A
"soft link" delays the binding until the name is accessed.  An
"automounter" adds a further form in which the binding of name to
filesystem is delayed until the name is accessed.

   The target volume, in its general form, is a tuple (host, filesystem,
sublink) which can be used to name the physical location of any volume
in the network.

   When a target is referenced, Amd ignores the sublink element and
determines whether the required filesystem is already mounted.  This is
done by computing the local mount point for the filesystem and checking
for an existing filesystem mounted at the same place.  If such a
filesystem already exists then it is assumed to be functionally
identical to the target filesystem.  By default there is a one-to-one
mapping between the pair (host, filesystem) and the local mount point so
this assumption is valid.

Operational Principles
======================

   Amd operates by introducing new mount points into the namespace.
These are called "automount" points.  The kernel sees these automount
points as NFS filesystems being served by Amd.  Having attached itself
to the namespace, Amd is now able to control the view the rest of the
system has of those mount points.  RPC calls are received from the
kernel one at a time.

   When a "lookup" call is received Amd checks whether the name is
already known.  If it is not, the required volume is mounted.  A
symbolic link pointing to the volume root is then returned.  Once the
symbolic link is returned, the kernel will send all other requests
direct to the mounted filesystem.

   If a volume is not yet mounted, Amd consults a configuration
"mount-map" corresponding to the automount point.  Amd then makes a
runtime decision on what and where to mount a filesystem based on the
information obtained from the map.

   Amd does not implement all the NFS requests; only those relevant to
name binding such as "lookup", "readlink" and "readdir".  Some other
calls are also implemented but most simply return an error code; for
example "mkdir" always returns "read-only filesystem".

Mounting a Volume
=================

   Each automount point has a corresponding mount map.  The mount map
contains a list of key-value pairs.  The key is the name of the volume
to be mounted.  The value is a list of locations describing where the
filesystem is stored in the network.  In the source for the map the
value would look like

     location1  location2  ...  locationN

   Amd examines each location in turn.  Each location may contain
"selectors" which control whether Amd can use that location.  For
example, the location may be restricted to use by certain hosts.  Those
locations which cannot be used are ignored.

   Amd attempts to mount the filesystem described by each remaining
location until a mount succeeds or Amd can no longer proceed.  The
latter can occur in three ways:

   * If none of the locations could be used, or if all of the locations
     caused an error, then the last error is returned.

   * If a location could be used but was being mounted in the
     background then Amd marks that mount as being "in progress" and
     continues with the next request; no reply is sent to the kernel.

   * Lastly, one or more of the mounts may have been "deferred".  A
     mount is deferred if extra information is required before the
     mount can proceed.  When the information becomes available the
     mount will take place, but in the mean time no reply is sent to
     the kernel.  If the mount is deferred, Amd continues to try any
     remaining locations.

   Once a volume has been mounted, Amd establishes a "volume mapping"
which is used to satisfy subsequent requests.

Automatic Unmounting
====================

   To avoid an ever increasing number of filesystem mounts, Amd removes
volume mappings which have not been used recently.  A time-to-live
interval is associated with each mapping and when that expires the
mapping is removed.  When the last reference to a filesystem is removed,
that filesystem is unmounted.  If the unmount fails, for example the
filesystem is still busy, the mapping is re-instated and its
time-to-live interval is extended.  The global default for this grace
period is controlled by the "-w" command-line option (see -w)
or the amd.conf parameter `dismount_interval' (see
dismount_interval Parameter.).  It is also possible to set this value
on a per-mount basis (see opts).

   Filesystems can be forcefully timed out using the Amq command.
See Run-time Administration.

Keep-alives
===========

   Use of some filesystem types requires the presence of a server on
another machine.  If a machine crashes then it is of no concern to
processes on that machine that the filesystem is unavailable.  However,
to processes on a remote host using that machine as a fileserver this
event is important.  This situation is most widely recognized when an
NFS server crashes and the behavior observed on client machines is that
more and more processes hang.  In order to provide the possibility of
recovery, Amd implements a "keep-alive" interval timer for some
filesystem types.  Currently only NFS makes use of this service.

   The basis of the NFS keep-alive implementation is the observation
that most sites maintain replicated copies of common system data such as
manual pages, most or all programs, system source code and so on.  If
one of those servers goes down it would be reasonable to mount one of
the others as a replacement.

   The first part of the process is to keep track of which fileservers
are up and which are down.  Amd does this by sending RPC requests to the
servers' NFS `NullProc' and checking whether a reply is returned.
While the server state is uncertain the requests are re-transmitted at
three second intervals and if no reply is received after four attempts
the server is marked down.  If a reply is received the fileserver is
marked up and stays in that state for 30 seconds at which time another
NFS ping is sent.

   Once a fileserver is marked down, requests continue to be sent every
30 seconds in order to determine when the fileserver comes back up.
During this time any reference through Amd to the filesystems on that
server fail with the error "Operation would block".  If a replacement
volume is available then it will be mounted, otherwise the error is
returned to the user.

   Although this action does not protect user files, which are unique on
the network, or processes which do not access files via Amd or already
have open files on the hung filesystem, it can prevent most new
processes from hanging.

   By default, fileserver state is not maintained for NFS/TCP mounts.
The remote fileserver is always assumed to be up.

Non-blocking Operation
======================

   Since there is only one instance of Amd for each automount point,
and usually only one instance on each machine, it is important that it
is always available to service kernel calls.  Amd goes to great lengths
to ensure that it does not block in a system call.  As a last resort
Amd will fork before it attempts a system call that may block
indefinitely, such as mounting an NFS filesystem.  Other tasks such as
obtaining filehandle information for an NFS filesystem, are done using a
purpose built non-blocking RPC library which is integrated with Amd's
task scheduler.  This library is also used to implement NFS keep-alives
(see Keep-alives.).

   Whenever a mount is deferred or backgrounded, Amd must wait for it
to complete before replying to the kernel.  However, this would cause
Amd to block waiting for a reply to be constructed.  Rather than do
this, Amd simply "drops" the call under the assumption that the kernel
RPC mechanism will automatically retry the request.

Supported Platforms
*******************

   Am-utils has been ported to a wide variety of machines and operating
systems.  Am-utils's code works for little-endian and big-endian
machines, as well as 32 bit and 64 bit architectures.  Furthermore, when
am-utils to an Operating System on one architecture, it is generally
readily portable to the same Operating System on all platforms on which
it is available.

   The table below lists those platforms supported by the latest
release.  The listing is based on the standard output from GNU's
`config.guess' script.  Since significant changes have been made to
am-utils, not all systems listed here have been verified working for all
features.

   Auto-Configured System Name Config Compile Amd NFS3 Shlib Hlfsd

   Auto-Configured System Name  Config  Compile  Amd  NFS3  Shlib  Hlfsd

   alpha-dec-osf4.0  yes      yes    yes   yes

   hppa1.1-hp-hpux10.10  yes      yes    yes   n/a    no

   hppa1.1-hp-hpux10.20  yes      yes    yes   n/a    no

   hppa1.1-hp-hpux9.01  yes      yes    yes   n/a

   hppa1.1-hp-hpux9.05  yes      yes    yes   n/a

   hppa1.1-hp-hpux9.07  yes      yes    yes   n/a

   i386-pc-bsdi2.1  yes      yes    yes   n/a

   i386-pc-bsdi3.0  yes      yes    yes   n/a

   i386-pc-solaris2.5.1  yes      yes    yes   yes    yes   yes