Add the tests(7) manual page, which describes why and how to run the
test suite and how to configure it.
This commit is contained in:
jmmv
parent
0af7f6db64
commit
929072d8ed
|
|
@ -1,10 +1,11 @@
|
|||
# $NetBSD: Makefile,v 1.21 2007/03/02 11:28:16 wiz Exp $
|
||||
# $NetBSD: Makefile,v 1.22 2010/06/26 11:15:27 jmmv Exp $
|
||||
# @(#)Makefile 8.1 (Berkeley) 6/5/93
|
||||
|
||||
# missing: eqnchar.7 man.7 ms.7 term.7
|
||||
|
||||
MAN= ascii.7 environ.7 hier.7 hostname.7 intro.7 mailaddr.7 \
|
||||
nls.7 operator.7 pkgsrc.7 release.7 \
|
||||
script.7 setuid.7 signal.7 sticky.7 symlink.7 sysctl.7
|
||||
script.7 setuid.7 signal.7 sticky.7 symlink.7 sysctl.7 \
|
||||
tests.7
|
||||
|
||||
.include <bsd.man.mk>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,172 @@
|
|||
.\" $NetBSD: tests.7,v 1.1 2010/06/26 11:15:27 jmmv Exp $
|
||||
.\"
|
||||
.\" Copyright (c) 2010 The NetBSD Foundation, Inc.
|
||||
.\" All rights reserved.
|
||||
.\"
|
||||
.\" Redistribution and use in source and binary forms, with or without
|
||||
.\" modification, are permitted provided that the following conditions
|
||||
.\" are met:
|
||||
.\" 1. Redistributions of source code must retain the above copyright
|
||||
.\" notice, this list of conditions and the following disclaimer.
|
||||
.\" 2. Redistributions in binary form must reproduce the above copyright
|
||||
.\" notice, this list of conditions and the following disclaimer in the
|
||||
.\" documentation and/or other materials provided with the distribution.
|
||||
.\"
|
||||
.\" THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND
|
||||
.\" CONTRIBUTORS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES,
|
||||
.\" INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
|
||||
.\" MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
|
||||
.\" IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS BE LIABLE FOR ANY
|
||||
.\" DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
||||
.\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE
|
||||
.\" GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
||||
.\" INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER
|
||||
.\" IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
|
||||
.\" OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN
|
||||
.\" IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
.\"
|
||||
.Dd June 26, 2010
|
||||
.Dt TESTS 7
|
||||
.Os
|
||||
.Sh NAME
|
||||
.Nm tests
|
||||
.Nd An introduction to the NetBSD test suite
|
||||
.Sh DESCRIPTION
|
||||
The
|
||||
.Nx
|
||||
test suite provides a collection of automated tests for two major purposes.
|
||||
On the one hand, the test suite aids
|
||||
.Em developers
|
||||
in catching bugs and regressions in the code when they performing modifications
|
||||
to the source tree.
|
||||
On the other hand, the test suite allows
|
||||
.Em end users
|
||||
(and, in particular, system administrators) to verify that fresh installations
|
||||
of the
|
||||
.Nx
|
||||
operating system behave correctly in their hardware platform and also to ensure
|
||||
that the system does not suffer from regressions during regular system
|
||||
operation and maintenance.
|
||||
.Pp
|
||||
The
|
||||
.Nx
|
||||
tests are implemented using the
|
||||
.Em Automated Testing Framework (ATF) ,
|
||||
a third-party package shipped with
|
||||
.Nx ;
|
||||
see
|
||||
.Xr atf 7
|
||||
for details.
|
||||
The
|
||||
.Nx
|
||||
test suite is distributed as a separate installation set, named
|
||||
.Pa tests.tgz ,
|
||||
and the test programs are all installed under the
|
||||
.Pa /usr/tests
|
||||
hieararchy.
|
||||
.Pp
|
||||
This manual page describes how to execute the test suite and how to configure
|
||||
some of its optional features.
|
||||
.Ss When to run the tests?
|
||||
Before diving into the details of how to run the test suite, here are some
|
||||
scenarios in which you should be running them:
|
||||
.Bl -bullet -offset indent
|
||||
.It
|
||||
After a fresh installation of
|
||||
.Nx
|
||||
to ensure that the system works correctly on your hardware platform.
|
||||
.It
|
||||
After an upgrade of
|
||||
.Nx
|
||||
to a different version to ensure that the new code works well on your
|
||||
hardware platform and that the upgrade did not introduce regressions in your
|
||||
configuration.
|
||||
.It
|
||||
After performing changes to the source tree to catch any bugs and/or regressions
|
||||
introduced by the modifications.
|
||||
.It
|
||||
Periodically, maybe from a
|
||||
.Xr cron 8
|
||||
job, to ensure that any changes to the system (such as the installation of
|
||||
third-party packages or manual modifications to configuration files) do not
|
||||
introduce unexpected failures.
|
||||
.El
|
||||
.Ss Running the tests
|
||||
Use the following commands to run the whole test suite:
|
||||
.Bd -literal -offset indent
|
||||
$ cd /usr/tests
|
||||
$ atf-run | atf-report
|
||||
.Ed
|
||||
.Pp
|
||||
The above will go through all test programs in
|
||||
.Pa /usr/tests
|
||||
recursive, execute them, and, at the very end, show a report of the results of
|
||||
the test suite.
|
||||
These results include the count of tests that succeeded (passed), the names of
|
||||
the tests that failed, and the count of the tests that were not executed
|
||||
(skipped) because the system configuration did not meet their requirements.
|
||||
.Pp
|
||||
If you are interested in saving the whole output of the test suite execution so
|
||||
that you can later investigate failures, use the following idiom instead:
|
||||
.Bd -literal -offset indent
|
||||
$ cd /usr/tests
|
||||
$ atf-run | tee ~/tests.log | atf-report
|
||||
.Ed
|
||||
.Pp
|
||||
The above command will save the raw output of the test suite in
|
||||
.Pa ~/tests.log ,
|
||||
which you can later inspect manually to look for failures.
|
||||
Note that the file contains a copy of the
|
||||
.Sq stdout
|
||||
and
|
||||
.Sq stderr
|
||||
of each test case, which becomes valuable during debugging.
|
||||
.Pp
|
||||
It is also possible to restrict which tests to execute so that only a small
|
||||
subsystem is tested; see
|
||||
.Xr atf-run 1
|
||||
for details.
|
||||
Additionally, it is also possible to run the test programs themselves by hand;
|
||||
see
|
||||
.Xr atf-test-program 1
|
||||
for more details, but be aware that you should only be doing this if you are
|
||||
debugging failing tests.
|
||||
.Ss Configuring the tests
|
||||
Some test cases in the
|
||||
.Nx
|
||||
test suite require the administrator to manually set up some configuration
|
||||
properties before they can run.
|
||||
Unless these properties are defined, the tests that require them will be marked
|
||||
as skipped and thus they will not be really executed.
|
||||
.Pp
|
||||
Each test suite is configured through a separate file that lives under
|
||||
.Pa /etc/atf/
|
||||
and that carries the name of the test suite.
|
||||
Henceforth, to configure the properties that affect the execution of the
|
||||
.Nx
|
||||
test suite, you need to edit
|
||||
.Pa /etc/atf/NetBSD.conf .
|
||||
These files conform to the configuration file format described in
|
||||
.Xr atf-formats 5 .
|
||||
.Pp
|
||||
The following properties are recognized:
|
||||
.Bl -tag -offset indent -width unprivilegedXuserXX
|
||||
.It Va unprivileged_user
|
||||
Specifies the name of a local, unprivileged user, that will be used by those
|
||||
tests that need to perform checks as non-root.
|
||||
.El
|
||||
.Sh SEE ALSO
|
||||
.Xr atf 7
|
||||
.Sh HISTORY
|
||||
The
|
||||
.Xr tests 7
|
||||
manual page first appeared in
|
||||
.Nx 6.0 .
|
||||
.Pp
|
||||
The ATF testing framework was first distributed with
|
||||
.Nx 5.0
|
||||
and the collection of test programs in
|
||||
.Pa /usr/tests
|
||||
has been growing since then.
|
||||
.Sh AUTHORS
|
||||
.An Julio Merino Aq jmmv@NetBSD.org
|
||||
Loading…
Reference in New Issue