Commit Graph

161 Commits

Author SHA1 Message Date
mrg
5d873ca56c make 'clean' and/or 'cleandir' actually clean all the things. 2018-05-31 09:08:25 +00:00
kamil
410141583e Improve wording for non-native speakers of English
Improve wording in t_ubsan tests in for c++.

No functional change intended.

Follow the changed in cc/ by <martin>
2018-05-04 12:51:18 +00:00
kamil
1560a24938 Explain VLA in c++/t_ubsan_vla_out_of_bounds.sh (ATF test)
VLA - Variable Length Array

Requested by <martin>
2018-05-04 12:27:58 +00:00
kamil
15009a7646 Explain VLA in t_ubsan_vla_out_of_bounds.sh (ATF test)
VLA - Variable Length Array

Requested by <martin>
2018-05-04 12:26:53 +00:00
martin
d3941ea9b2 Use more formal speech in messages 2018-05-04 10:44:24 +00:00
kamil
e308425ef5 Add new ATF Undefined Behavior Sanitizer tests
Add new cc and c++ tests to check whether UBSan works.
These tests are prepared for GCC (in base) and Clang (with external patches).

Enable these tests for all ports by default, just verify whether we are
using GCC/Clang or a compatible compiler.

Add five equivalent C and C++ tests:
 - Integer addition overflow
 - Integer divide by zero
 - Integer negation overflow
 - Integer subtraction overflow
 - VLA out of bounds

All tests pass on NetBSD/amd64.

Patch submitted by <Harry Pantazis>
Minor cleanup by <myself>
2018-05-02 18:46:05 +00:00
kamil
8a7b75cec0 Import new C and C++ ATF tests for ASan
Add new tests:
 - tests/usr.bin/cc/t_asan_poison.sh
 - tests/usr.bin/c++/t_asan_poison.sh

These tests verify the following build options:
 - regular
 - profile
 - pic
 - pie
 - compat32
 - (static unsupported)

These tests verify whether ASan code can include compiler and sanitizer
specific header: <sanitizer/asan_interface.h>. The testing code checks
the ASAN_POISON_MEMORY_REGION() functionality, poisoning valid memory and
asserting that it triggers expected failure.

Patch submitted by <Siddharth Muralee>
2018-04-11 03:25:25 +00:00
martin
b29a640aff Use tcpdump -n to avoid reverse DNS lookup. There is no point in having
different results from running this test in a networked vs. a non-networked
environment.
Adjust golden output accordingly.
Fixes PR 53150.
2018-04-09 16:21:05 +00:00
kamil
8e4f836c68 Add new ATF tests for Address Sanitzier (ASan)
Add new C and C++ tests:
 - t_asan_double_free
 - t_asan_global_buffer_overflow
 - t_asan_heap_overflow
 - t_asan_off_by_one
 - t_asan_uaf

Each tests checks:
 - regular build
 - 32-bit
 - PIC
 - PIE
 - profile

These tests require paxctl(8) to disable ASLR in order to work in a
predictable way. This is especially true for all !regular builds with
additional compiler flags.

There are no static variations of these tests as this mode is not supported
in upstream ASan.

Enable these tests on amd64 and i386.

This is part two patch, adding the remaining C++ changes.

Patch submitted by <Siddharth Muralee>
Additional polishing by myself.
2018-04-04 23:53:26 +00:00
kamil
9f0bdbd997 Add new ATF tests for Address Sanitzier (ASan)
Add new C and C++ tests:
 - t_asan_double_free
 - t_asan_global_buffer_overflow
 - t_asan_heap_overflow
 - t_asan_off_by_one
 - t_asan_uaf

Each tests checks:
 - regular build
 - 32-bit
 - PIC
 - PIE
 - profile

These tests require paxctl(8) to disable ASLR in order to work in a
predictable way. This is especially true for all !regular builds with
additional compiler flags.

There are no static variations of these tests as this mode is not supported
in upstream ASan.

Enable these tests on amd64 and i386.

Patch submitted by <Siddharth Muralee>
Additional polishing by myself.
2018-04-04 23:51:35 +00:00
kamil
b4fcb5aedb Correct a small regression typo in usr.bin/c++ ATF tests
TESTS_SH= changed to TESTS_SH+=
2018-04-04 15:53:04 +00:00
kamil
b269e95d9e Sort entries in ATF tests in tests/usr.bin/c++/Makefile
No functional change intended.
2018-04-04 15:08:59 +00:00
kamil
77dc83fa09 Add new C++ ATF tests
Add new variations for existing C++ tests:
 - Static
 - Profile+32-bit
 - PIC+32-bit
 - PIC+Profile
 - PIC+Profile+32-bit

All tests pass for NetBSD/amd64:
 - t_cxxruntime
 - t_hello
 - t_static_destructor

+--------------------------------------------------------------+
| Options            | cxxruntime | hello  | static_destructor |
+--------------------------------------------------------------+
| None               | Passed     | Passed | Passed            |
| 32-bit             | Passed     | Passed | Passed            |
| PIC                | Passed     | Passed | Passed            |
| PIE                | Passed     | Passed | Passed            |
| Profile            | Passed     | Passed | Passed            |
| Static             | Passed     | Passed | Passed            |
| Profile+32-bit     | Passed     | Passed | Passed            |
| PIC+32-bit         | Passed     | Passed | Passed            |
| PIC+Profile        | Passed     | Passed | Passed            |
| PIC+Profile+32-bit | Passed     | Passed | Passed            |
+--------------------------------------------------------------+

Add new C++11 std::call_once tests:
 - t_call_once
 - t_call_once2

Add new C++11 test with pthread_once(3) and C++ lambda:
 - t_pthread_once

All tests with the profile option for std::call_once and
pthread_once(3) are marked as expected failure (NetBSD/amd64).

Results for *_once*:
+------------------------------------------------------------+
| Options            | call_once | call_once2 | pthread_once |
+------------------------------------------------------------+
| None               | Passed    | Passed     | Passed       |
| 32-bit             | Passed    |  Passed    | Passed       |
| PIC                | Passed    | Passed     | Passed       |
| PIE                | Passed    | Passed     | Passed       |
| Profile            |  Failed   |  Failed    |  Failed      |
| Static             | Passed    | Passed     | Passed       |
| Profile+32-bit     |  Failed   |  Failed    |  Failed      |
| PIC+32-bit         | Passed    | Passed     | Passed       |
| PIC+Profile        |  Failed   |  Failed    |  Failed      |
| PIC+Profile+32-bit |  Failed   |  Failed    |  Failed      |
+------------------------------------------------------------+

Long term there is an option to refacotr the framework for C and C++ tests,
in order to reduce code duplication.

Patches sent by Yang Zheng <tomsun.0.7@gmail.com>
2018-03-24 00:26:51 +00:00
kre
79eedfcd84 Do the previous test a better way - for a file, test is generally
adequate, but for a device, we really need to actually try opening
it to determine that it is possible - so do the test that way, then
if the open succeeds once, assume it will the second time (which then
holds it open.)
2017-07-25 22:28:22 +00:00
kre
ddab1f6e3b Correct oversight in previous ... redirecting into a compound statement
causes the shell to exit if the redirect fails (posix says "may exit"
and /bin/sh does - maybe should give that more thought) - which will
happen if /dev/pad0 does not exist, causing a very messy test abort
(the shell running the test is not supposed to just go away).  So
check tha the device exista and is readable before attempting to open it.

Problem brought to my attention by nat@ - thanks...
2017-07-25 21:25:03 +00:00
kre
ae98bc7d08 NFC: Typo in a comment corrected. 2017-07-18 13:17:37 +00:00
kre
a94bbc243b Make sure that the open of /dev/pad0 has succeeded (or at the very
least been attempted) before attempting to open /dev/mixer to determine
if the system being tested has audio or not.  Leaving this for the background
cat command to do causes a race between that command and the parent sh.
Move this code to a helper function to avoid duplicating it.

Also avoid attempting to kill the background cat if it was never created.
The kill is likely unnecessary anyway, ATF seems to clean up processes
like that one without assistance.   Which is a good thing, as the kill
does not happen if the test is skipped, or fails.
2017-07-18 12:40:57 +00:00
nat
27ecfe9c27 As pad devices are now created on demand - pad has to be open for a
corresponding mixer to be available.
2017-07-03 09:08:35 +00:00
martin
b40441be27 In the -m32 test, additionally test profiled binaries. 2017-05-31 11:08:35 +00:00
martin
4fcb013cbf Add (currently failing) test cases for profiled programs. 2017-05-18 10:29:47 +00:00
kamil
d1be05c0a9 Add new ATF C++ tests in usr.bin/c++: t_static_destructor
These tests are cloned from t_cxxruntime and check proper order of destructor
calls. They must be reported in reverse order of constructor completion.

Added tests:
 - static_destructor
 - static_destructor_pic
 - static_destructor_pie
 - static_destructor32

This test file replaces src/regress/usr.bin/c++/static_destructor.
2017-05-14 02:02:25 +00:00
kamil
179f19cf82 Use more C++ sugar in t_cxxruntime (ATF test) 2017-05-14 01:13:44 +00:00
kamil
9fab767d88 Add new ATF C++ tests in usr.bin/c++
These tests are cloned from t_hello and use c++ runtime basic functions.

Added tests:
 - hello
 - hello_pic
 - hello_pie
 - hello32
2017-05-14 01:03:21 +00:00
kamil
b971227c60 Add new c++ ATF tests in usr.bin/c++: t_hello
This is a copy of t_hello from usr.bin/cc.

Added tests:
 - hello
 - hello_pic
 - hello_pie
 - hello32

These tests do not use c++ runtime library functions.

Protect these tests with MKCXX.
2017-05-14 00:07:07 +00:00
kamil
4dde76f99d Correct hellp_pic ATF test in usr.bin/cc
Drop "-dPIC", this is misspelled "-DPIC" option for libtool.
Enhance code example to be more pedantic.
2017-05-13 23:51:39 +00:00
kre
dfcb546256 If we are using the pad audio device, there must be a process with
the corresponding pad device open, or we get EIO from audio accesses

Explained and fix provided by Nathanial Sloss <nat@n.o>

Note: if we are testing and using real audio hardware, the open
of /dev/pad0 is irrelevant (but harmless, so we don't attempt to
check) and what's more it doesn't matter if it succeeds or fails.

If we're testing under qemu (or any other situation where the only
audio "hardware" is pad) then the open will work, and there should be
no more EIO.

If there is no audio hardware of any kind on the system being tested,
the attempt top open /dev/mixer should fail, and the test will be
skipped.
2017-04-20 00:17:11 +00:00
christos
48c8a60c2e new test 2017-03-06 23:07:42 +00:00
christos
47f9cf0422 add builtin_offsetof 2017-03-06 21:02:47 +00:00
christos
b811ddbd72 add a test for a typeof cast 2017-03-06 12:00:27 +00:00
kre
71b606ef4e Drop the test for QEMU and instead skip relevant tests when /dev/mixer
can't be opened (which more accurately reflects when mixerctl is going
to fail...)

Based upon an idea from Andreas Gustafsson (gson@) - except that using
/dev/audio0 for this purpose doesn't work, if the only audio device
configured is pad0 an open of audio fails with EIO (???)

While here, perpare for the updated mixerctl coming soon to a repository
near you...
	Use correct usage in the test of a bogus -d arg (otherwise the
		new mixerctl will complain about usage, and never even
		attempt to open the bogus device)
	Don't require /dev/mixer for the noargs -> generate usage msg
		test ... this will now generate a failing test with the
		old mixerctl if there is no working /dev/mixer, but
		that mixerctl won't be around much longer.
2017-02-23 14:01:37 +00:00
kre
700ea9d267 Limit previous to the i386 qemu kernels, the tests work on amd64 and sparc
(which have some audio configured.)
2017-02-23 02:28:10 +00:00
kre
e4eeed67d9 Skip most of the mixerctl tests when running under QEMU (assumed to be for ATF)
as the kernel that runs them has no audio (and no mixers) configured.

The usage message test might be returned some day if /usr/bin/mixerctl
gets modified so it doesn't attempt to open the device (and error out)
in cases where the device isn't actually going to be used (and -d wasn't
given to set the device name explicitly).
2017-02-23 02:09:24 +00:00
kre
697c0e2acf Ugh... fix a last minute fix that went horribly wrong! 2017-02-21 21:22:45 +00:00
kre
67cef682dd PR bin/50934
Add a test program for the bug described in this PR.
This is the first pkill/pgrep/prenice test (more would be good!)

This test has been confirmed to work once the bug described in the PR
has been fixed, so the test is not marked "expected to fail" even
though initially that is what should happen.

Note: the test cana also fail if the system running the tests happens
to be running processes with names that match the patterns searched for
by the test, other than the test program itself.  This is expected to be
unlikely.
2017-02-21 10:40:30 +00:00
christos
8684dbb301 PR/51875: Ngie Cooper: portability fixes 2017-01-14 20:43:52 +00:00
christos
f0970fd0b2 mixerctl tests from Charlotte Koch 2017-01-02 15:40:09 +00:00
abhinav
6cddff1a99 Add tests for uniq(1) based on the example test cases provided in the POSIX man page.
Ok christos
2016-10-22 14:13:39 +00:00
christos
eb8cf8dd65 house-cleaning 2016-08-28 15:59:15 +00:00
christos
dd0e0080d2 adjust to current config format 2016-08-27 12:08:14 +00:00
christos
080051aa7f Add union cast test 2016-08-19 10:21:50 +00:00
christos
fa75a57426 new test for struct that only has anon members, and that the size of struct
is computer right.
2016-08-18 14:45:56 +00:00
agc
83432fa6b8 remove some unused test cases 2016-06-01 14:52:56 +00:00
agc
6d27eeee35 Use human2atf to generate a t_netpgpverify.sh script which is half the
size of the previous one (using on-the-fly gzip compression), and includes
more cases (including a gpg-signed file for cross-testing purposes).

Add the appropriate Testspec file.
2016-06-01 06:39:11 +00:00
gson
5973801c0c Add regression tests for some past gdb bugs. 2016-04-08 10:09:16 +00:00
christos
a34d3aeabd Add test for PR/49872 2016-04-05 00:48:53 +00:00
gson
43ed5fd66c Fix typo 2016-04-03 14:41:30 +00:00
christos
40c99df3a6 Add an anonymous struct test 2015-10-14 16:32:55 +00:00
uebayasi
8b6b28e153 Catch up with the current, generated Makefile format. 2015-10-04 07:59:47 +00:00
christos
862f3563b3 new test 2015-07-29 18:26:15 +00:00
christos
198668b6a8 new c99 init test found by xorg's Iconify.c 2015-07-28 18:05:19 +00:00