Catch and report unhandled exceptions before they propagate to C land
The C++ interface to run test cases goes like this:
1) C++ run function -> 2) C run function -> 3) C++ wrapper for
test case -> 4) test case head/body/cleanup
The previous code caught and reported unhandled exceptions in 1).
However, such approach does not seem to work everywhere. It fails,
for example, in NetBSD/i386 but works in NetBSD/amd64. I am not sure
which platform implementation is correct nor if there even _is_ a
defined behavior. No matter what, it feels wrong and clunky.
Move the last-resort exception catching to happen in 3) so that
exceptions don't propagate back to C.
Fixes the test-programs/result_test:result_exception test case in
NetBSD/i386 5.99.34.
Miscellaneous features
* Added expected failures support to test cases and atf-run. These
include, for example, expected clean exits, expected reception of fatal
signals, expected timeouts and expected errors in condition checks.
These statuses can be used to denote test cases that are known to fail
due to a bug in the code they are testing. atf-report reports these
tests separately but they do not count towards the failed test cases
amount.
* Added the ATF_CHECK_ERRNO and ATF_REQUIRE_ERRNO to the C library to
allow easy checking of call failures that update errno.
* Added the has.cleanup meta-data property to test caes that specifies
whether the test case has a cleanup routine or not; its value is
automatically set. This property is read by atf-run to know if it has to
run the cleanup routine; skipping this run for every test case
significantly speeds up the run time of test suites.
* Reversed the order of the ATF_CHECK_THROW macro in the C++ binding to
take the expected exception as the first argument and the statement to
execute as the second argument.
Changes in atf-check
* Changed atf-check to support negating the status and output checks by
prefixing them with not- and added support to specify multiple checkers
for stdout and stderr, not only one.
* Added the match output checker to atf-check to look for regular
expressions in the stdout and stderr of commands.
* Modified the exit checks in atf-check to support checking for the
reception of signals.
Code simplifications and cleanups
* Removed usage messages from test programs to simplify the
implementation of every binding by a significant amount. They just now
refer the user to the appropriate manual page and do not attempt to wrap
lines on terminal boundaries. Test programs are not supposed to be run
by users directly so this minor interface regression is not important.
* Removed the atf-format internal utility, which is unused after the
change documented above.
* Removed the atf-cleanup internal utility. It has been unused since the
test case isolation was moved to atf-run in 0.8
* Splitted the Makefile.am into smaller files for easier maintenance and
dropped the use of M4. Only affects users building from the repository
sources.
* Intermixed tests with the source files in the source tree to provide
them more visibility and easier access. The tests directory is gone from
the source tree and tests are now suffixed by _test, not prefixed by t_.
* Simplifications to the atf-c library: removed the io, tcr and ui
modules as they had become unnecessary after all simplifications
introduced since the 0.8 release.
* Removed the application/X-atf-tcr format introduced in 0.8 release.
Tests now print a much simplified format that is easy to parse and nicer
to read by end users. As a side effect, the default for test cases is
now to print their results to stdout unless otherwise stated by providing
the -r flag.
* Removed XML distribution documents and replaced them with plain-text
documents. They provided little value and introduced a lot of complexity
to the build system.
* Simplified the output of atf-version by not attempting to print a
revision number when building form a distfile. Makes the build system
easier to maintain.
- Recognize -C for pkg_add like the rest of the tools do
- Do the existing check for pkg_add -f, it makes the combination of
-f and -U a bit less surprising
- Fix option handling for CHECK_VULNERABILITIES (from spz@)
- Make end-of-life check the default. pkg_install.conf(5) can be used
to override the default. The existing admin -e & co continue to work as
before.
- kill MKPIGZ.
- add MKPIGZGZIP. if it is "no", then only install pigz as pigz. if it is
not "no", don't install mrgzip, install pigz as gzip.
in all cases, the z* scripts.
i don't like the name MKPIGZGZIP so if anyone has a better name, feel free
to replace it. i don't care enough since killing the ugly name is just
more incentive to get pigz doing .bz2, .Z and .z.
- Improved before-queue content filter performance. With
"smtpd_proxy_options = speed_adjust", the Postfix SMTP server
receives the entire message before it connects to a before-queue
content filter. Typically, this allows Postfix to handle the same
mail load with fewer content filter processes.
- Improved address verification performance. The verify database is now
persistent by default, and it is automatically cleaned periodically. Under
overload conditions, the Postfix SMTP server no longer waits up to 6 seconds
for an address probe to complete.
- Support for reputation management based on the local SMTP client IP address.
This is typically implemented with "FILTER transportname:" actions in access
maps or header/body checks, and mail delivery transports in master.cf with
unique smtp_bind_address values.
initial import of pigz 2.1.6 sources.
from http://www.zlib.net/pigz/:
"""
pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the
hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.
How is it pronounced?
I'm glad you asked. It is pronounced "pig-zee". It is not pronounced like the plural of pig.
"""
once i am done adding bz2 support to pigz, i plan to obsolete my usr.bin/gzip.
from http://www.zlib.net/pigz/:
"""
pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. pigz was written by Mark Adler, and uses the zlib and pthread libraries.
How is it pronounced?
I'm glad you asked. It is pronounced "pig-zee". It is not pronounced like the plural of pig.
"""
once i am done adding bz2 support to pigz, i plan to obsolete my usr.bin/gzip.
which are known to fail, e.g.:
atf_tc_set_md_var(tc, "xfail", "PR kern/43456");
Expected failures do not count towards the ultimate pass/fail result
from the test run:
pain-rustique:39:~/<2>src/tests/fs/ptyfs> atf-run t_nullpts | atf-report
Tests root: /home/pooka/src/wholesrc2/src/tests/fs/ptyfs
t_nullpts (1/1): 1 test cases
nullrevoke: Expected failure: PR kern/43456
Summary for 1 test programs:
0 passed test cases.
0 failed test cases.
1 expected failures.
0 skipped test cases.
pain-rustique:40:~/<2>src/tests/fs/ptyfs> echo $?
0
However, an xfail test which passes will count as a failure, i.e.
xfail inverts test case success/fail. This way we can get a better
sense from the ultimate verdict of the NetBSD atf run by seeing if
there were any unexpected failures, i.e. new regressions.
This feature will be present in the upcoming atf 0.10 release,
possibly with finer grained control.
patch reviewed by jmmv
----------
Fix detection of crashed test cases
Prevent cross-test-case contamination that occured when a first test case
passed and the second crashed. The second could pick up the result of the
first test case and not be reported as failed.
Similarly, change the way timed out test cases are reported back to the
caller. The creation of a temporary results file was just a really stupid
way of passing information around and introduced false positives if the
test case creates a results file before timing out.
Fixes ticket #35.
----------
Problem originally reported by pooka@.
jmmv