156 lines
5.2 KiB
Plaintext
156 lines
5.2 KiB
Plaintext
Automated Testing Framework (atf)
|
|
The NetBSD Foundation, Inc.
|
|
---------------------------------------------------------------------------
|
|
|
|
|
|
Version 0.5
|
|
===========
|
|
|
|
Release date: May 1st, 2008
|
|
Status: Experimental
|
|
|
|
* Clauses 3 and 4 of the BSD license used by the project were dropped.
|
|
All the code is now under a 2-clause BSD license compatible with the
|
|
GNU General Public License (GPL).
|
|
|
|
* Added a C-only binding so that binary test programs do not need to be
|
|
tied to C++ at all. This binding is now known as the atf-c library.
|
|
|
|
* Renamed the C++ binding to atf-c++ for consistency with the new atf-c.
|
|
|
|
* Renamed the POSIX shell binding to atf-sh for consistency with the new
|
|
atf-c and atf-c++.
|
|
|
|
* Added a -w flag to test programs through which it is possible to specify
|
|
the work directory to be used. This was possible in prior releases by
|
|
defining the workdir configuration variable (-v workdir=...), but was a
|
|
conceptually incorrect mechanism.
|
|
|
|
* Test programs now preserve the execution order of test cases when they
|
|
are given in the command line. Even those mentioned more than once are
|
|
executed multiple times to comply with the user's requests.
|
|
|
|
|
|
Version 0.4
|
|
===========
|
|
|
|
Release date: February 4th, 2008
|
|
Status: Experimental
|
|
|
|
Changes:
|
|
|
|
* Added two new manual pages, atf-c++-api and atf-sh-api, describing the
|
|
C++ and POSIX shell interfaces used to write test programs.
|
|
|
|
* Added a pkg-config file, useful to get the flags to build against the
|
|
C++ library or to easily detect the presence of ATF.
|
|
|
|
* Added a way for test cases to require a specific architecture and/or
|
|
machine type through the new 'require.arch' and 'require.machine'
|
|
meta-data properties, respectively.
|
|
|
|
* Added the 'timeout' property to test cases, useful to set an upper-bound
|
|
limit for the test's run time and thus prevent global test program stalls
|
|
due to the test case's misbehavior.
|
|
|
|
* Added the atf-exec(1) internal utility, used to execute a command after
|
|
changing the process group it belongs to.
|
|
|
|
* Added the atf-killpg(1) internal utility, used to kill process groups.
|
|
|
|
* Multiple portability fixes. Of special interest, full support for SunOS
|
|
(Solaris Express Developer Edition 2007/09) using the Sun Studio 12 C++
|
|
compiler.
|
|
|
|
* Fixed a serious bug that prevented atf-run(1) from working at all under
|
|
Fedora 8 x86_64. Due to the nature of the bug, other platforms were
|
|
likely affected too.
|
|
|
|
|
|
Version 0.3
|
|
===========
|
|
|
|
Release date: November 11th, 2007
|
|
Status: Experimental
|
|
|
|
Changes:
|
|
|
|
* Added XML output support to atf-report. This is accompanied by a DTD for
|
|
the format's structure and sample XSLT/CSS files to post-process this
|
|
output and convert it to a plain HTML report.
|
|
|
|
* Changed atf-run to add system information to the report it generates.
|
|
This is currently used by atf-report's XML output only, and is later
|
|
printed in the HTML reports in a nice and useful summary table. The user
|
|
and system administrator are allowed to tune this feature by means of
|
|
hooks.
|
|
|
|
* Removed the test cases' 'isolated' property. This was intended to avoid
|
|
touching the file system at all when running the related test case, but
|
|
this has not been true for a long while: some control files are
|
|
unconditionally required for several purposes, and we cannot easily get
|
|
rid of them. This way we remove several critical and delicate pieces of
|
|
code.
|
|
|
|
* Improved atf-report's CSV output format to include information about
|
|
test programs too.
|
|
|
|
* Fixed the tests that used atf-compile to not require this tool as a
|
|
helper. Avoids systems without build-time utilities to skip many tests
|
|
that could otherwise be run. (E.g. NetBSD without the comp.tgz set
|
|
installed.)
|
|
|
|
* Many general cleanups: Fixed many pieces of code marked as ugly and/or
|
|
incomplete.
|
|
|
|
|
|
Version 0.2
|
|
===========
|
|
|
|
Release date: September 20th, 2007
|
|
Status: Experimental
|
|
|
|
Changes:
|
|
|
|
* Test cases now get a known umask on entry.
|
|
|
|
* atf-run now detects many unexpected failures caused by test programs
|
|
and reports them as bogus tests. atf-report is able to handle these
|
|
new errors and nicely reports them to the user.
|
|
|
|
* All the data formats read and written by the tools have been documented
|
|
and cleaned up. These include those grammars that define how the
|
|
different components communicate with each other as well as the format
|
|
of files written by the developers and users: the Atffiles and the
|
|
configuration files.
|
|
|
|
* Added the atf-version tool, a utility that displays information about the
|
|
currently installed version of ATF.
|
|
|
|
* Test cases can now define an optional cleanup routine to undo their
|
|
actions regardless of their exit status.
|
|
|
|
* atf-report now summarizes the list of failed (bogus) test programs when
|
|
using the ticker output format.
|
|
|
|
* Test programs now capture some termination signals and clean up any
|
|
temporary files before exiting the program.
|
|
|
|
* Multiple bug fixes and improvements all around.
|
|
|
|
|
|
Version 0.1
|
|
===========
|
|
|
|
Release date: August 20th, 2007
|
|
Status: Experimental
|
|
|
|
Changes:
|
|
|
|
* First public version. This was released coinciding with the end of the
|
|
Google Summer of Code 2007 program.
|
|
|
|
|
|
---------------------------------------------------------------------------
|
|
End of document
|