Commit Graph

188 Commits

Author SHA1 Message Date
kamil 5612f2cac8 Define target_not_supported_body() in TSan, MSan and libFuzzer tests 2019-08-23 06:39:54 +00:00
kamil f0720e6929 Add ATF c and c++ tests for TSan, MSan, libFuzzer
These tests require Clang/LLVM 7 or newer on NetBSD.

Contributed by Yang Zheng during GSoC 2018.
2019-08-18 20:15:58 +00:00
kamil 825c3698b4 Avoid symbol clashes in tests/usr.bin/id under MKSANITIZER
Remove symbol conflicts for:

 - __getpwnam50
 - __getpwuid50
 - getgrgid
 - getgrouplist
 - getgroups
2019-08-15 08:22:52 +00:00
kre 311f349816 Stop assuming that printf handles options in any way at all
(it doesn't - that is, shouldn't) which includes processing -- as an
"end of options".  The first arg is (always) the format string.

Remove/fix tests that assumed the contrary.

Problem (with printf) pointed out on tech-userlevel by Thierry Laronde.
2019-07-21 15:25:59 +00:00
kre da8bc57f2b Correct a typo. It is atf_require_prog not atf_require_pfog ... 2019-07-21 15:00:18 +00:00
gson 1430c33c98 Add a regression test for PR bin/54154, or in other words, make sure
we get notified in a timely manner if gdb's XML support disappears
again.
2019-06-07 19:05:15 +00:00
christos af5402251e Add explanatory text 2019-05-15 13:43:45 +00:00
kamil d845d12b1a Upgrade indent(1)
Merge all the changes from the recent FreeBSD HEAD snapshot
into our local copy.

FreeBSD actively maintains this program in their sources and their
repository contains over 100 commits with changes.

Keep the delta between the FreeBSD and NetBSD versions to absolute
minimum, mostly RCS Id and compatiblity fixes.

Major chages in this import:

 - Added an option -ldi<N> to control indentation of local variable names.
 - Added option -P for loading user-provided files as profiles
 - Added -tsn for setting tabsize
 - Rename -nsac/-sac ("space after cast") to -ncs/-cs
 - Added option -fbs Enables (disables) splitting the function declaration and opening brace across two lines.
 - Respect SIMPLE_BACKUP_SUFFIX environment variable in indent(1)
 - Group global option variables into an options structure
 - Use bsearch() for looking up type keywords.
 - Don't produce unneeded space character in function declarators
 - Don't unnecessarily add a blank before a comment ends.
 - Don't ignore newlines after comments that follow braces.

Merge the FreeBSD intend(1) tests with our ATF framework.
All tests pass.

Upgrade prepared by Manikishan Ghantasala.
Final polishing by myself.

Part II, checkin new files.
2019-04-04 15:27:35 +00:00
kamil d6a1bc3f07 Upgrade indent(1)
Merge all the changes from the recent FreeBSD HEAD snapshot
into our local copy.

FreeBSD actively maintains this program in their sources and their
repository contains over 100 commits with changes.

Keep the delta between the FreeBSD and NetBSD versions to absolute
minimum, mostly RCS Id and compatiblity fixes.

Major chages in this import:

 - Added an option -ldi<N> to control indentation of local variable names.
 - Added option -P for loading user-provided files as profiles
 - Added -tsn for setting tabsize
 - Rename -nsac/-sac ("space after cast") to -ncs/-cs
 - Added option -fbs Enables (disables) splitting the function declaration and opening brace across two lines.
 - Respect SIMPLE_BACKUP_SUFFIX environment variable in indent(1)
 - Group global option variables into an options structure
 - Use bsearch() for looking up type keywords.
 - Don't produce unneeded space character in function declarators
 - Don't unnecessarily add a blank before a comment ends.
 - Don't ignore newlines after comments that follow braces.

Merge the FreeBSD intend(1) tests with our ATF framework.
All tests pass.

Upgrade prepared by Manikishan Ghantasala.
Final polishing by myself.
2019-04-04 15:22:13 +00:00
mrg 01ef24fd3f fix hello_profile's hello_profile_head function name. 2019-02-09 02:36:28 +00:00
mrg f4fc04f995 add $NetBSD$ tags. 2019-02-09 00:13:19 +00:00
mrg 9ca42b4c5e add very simple test that you can link -fopenmp.
currently fails, but should work soon...
2019-02-09 00:12:14 +00:00
mgorny de2592f9f0 Move common UBSAN test case code into ubsan_common.subr
Reviewed by <kamil>
2019-01-29 20:07:03 +00:00
mgorny ff295ecc8d Move common ASAN test case code into asan_common.subr
Reviewed by <kamil>
2019-01-29 20:02:34 +00:00
mgorny b4870a4524 Move common UBSAN test case code into ubsan_common.subr
Reviewed by <kamil>
2019-01-29 19:59:10 +00:00
mgorny bf6dc715c6 Move common ASAN test case code into asan_common.subr
Reviewed by <kamil>
2019-01-29 19:56:37 +00:00
christos 3f3dbbcd73 add a range and a repeat regex test. 2019-01-19 01:02:12 +00:00
christos f629fca860 This now works too. 2019-01-08 00:22:11 +00:00
christos 82a6fadf05 this now works... 2019-01-08 00:21:19 +00:00
kre 30a7129418 Move the tests of character value args to floating conversions
out of the %e test and into the NetBSD specific tests, while POSIX
once (apparently) required (or seemed to require) support for that,
it no longer does (and character value args to float conversions do
not make much sense - char valuse are always integers).

When char conversions are supported, test that \'  (or \" ) (that is,
no character following the quote) works as an arg (should result in zero,
as there's a \0 in the arg after the quote char).

Add some big/small number %g conversion tests.
2018-09-14 19:57:57 +00:00
kre fdc5bd383d When the shell being tested for builtin printf supports "type -t"
use it to test if printf is built in, it is must more reliable that
guessing at the output format and message of "type" (without -t).
2018-09-14 19:52:23 +00:00
kre 15c15cae0c Add a test where printf is run twice in the same shell, and the
first invocation uses \c in a %b arg - make sure that 2nd invocation
is not affected by that  (it was until recently, for a very long time).
2018-09-10 15:02:11 +00:00
kre ecfbb4457e Make sure we're in the standard locale when testing for printf
being built in to the shell, so we don't receive an unrecognisable
(to the script) message from the shell when we ask it.
2018-09-08 18:24:01 +00:00
kre d1e529d31c Add ATF tests for printf(1)
Two new test programs, one for the version of printf in /bin/sh
and one for the command /usr/bin/printf (t_builtin and t_command)

Each test program has 28 test cases (the same in each) of which
currently 27 pass, and 1 is skipped.

See the test scripts themselves for more information.
2018-09-05 21:05:40 +00:00
maya f4f9a0f4e4 amend test case to also check for -fprofile-generate
(clang also supports this flag)

PR toolchain/53567
2018-09-03 21:54:57 +00:00
kamil b4d61ce68e Disable profile and compat 32-bit c++ sanitizer tests
These tests are known to be faulty. These fatures aren't ready.
They tend to break to unrelated changes to the codebase.

Base Clang and GCC are in general dated to use with sanitizers as these
features are being fixed on HEAD versions of both compilers (mostly Clang).
2018-07-16 07:27:26 +00:00
kamil e77b94561b Disable profile and compat 32-bit tests cc sanitizer tests
These tests are known to be faulty. These fatures aren't ready.
They tend to break to unrelated changes to the codebase and require.

Base Clang and GCC are in general dated to use with sanitizers as these
features are being fixed on HEAD versions of both compilers (mostly Clang).
2018-07-16 07:25:58 +00:00
mrg 5d873ca56c make 'clean' and/or 'cleandir' actually clean all the things. 2018-05-31 09:08:25 +00:00
kamil 410141583e Improve wording for non-native speakers of English
Improve wording in t_ubsan tests in for c++.

No functional change intended.

Follow the changed in cc/ by <martin>
2018-05-04 12:51:18 +00:00
kamil 1560a24938 Explain VLA in c++/t_ubsan_vla_out_of_bounds.sh (ATF test)
VLA - Variable Length Array

Requested by <martin>
2018-05-04 12:27:58 +00:00
kamil 15009a7646 Explain VLA in t_ubsan_vla_out_of_bounds.sh (ATF test)
VLA - Variable Length Array

Requested by <martin>
2018-05-04 12:26:53 +00:00
martin d3941ea9b2 Use more formal speech in messages 2018-05-04 10:44:24 +00:00
kamil e308425ef5 Add new ATF Undefined Behavior Sanitizer tests
Add new cc and c++ tests to check whether UBSan works.
These tests are prepared for GCC (in base) and Clang (with external patches).

Enable these tests for all ports by default, just verify whether we are
using GCC/Clang or a compatible compiler.

Add five equivalent C and C++ tests:
 - Integer addition overflow
 - Integer divide by zero
 - Integer negation overflow
 - Integer subtraction overflow
 - VLA out of bounds

All tests pass on NetBSD/amd64.

Patch submitted by <Harry Pantazis>
Minor cleanup by <myself>
2018-05-02 18:46:05 +00:00
kamil 8a7b75cec0 Import new C and C++ ATF tests for ASan
Add new tests:
 - tests/usr.bin/cc/t_asan_poison.sh
 - tests/usr.bin/c++/t_asan_poison.sh

These tests verify the following build options:
 - regular
 - profile
 - pic
 - pie
 - compat32
 - (static unsupported)

These tests verify whether ASan code can include compiler and sanitizer
specific header: <sanitizer/asan_interface.h>. The testing code checks
the ASAN_POISON_MEMORY_REGION() functionality, poisoning valid memory and
asserting that it triggers expected failure.

Patch submitted by <Siddharth Muralee>
2018-04-11 03:25:25 +00:00
martin b29a640aff Use tcpdump -n to avoid reverse DNS lookup. There is no point in having
different results from running this test in a networked vs. a non-networked
environment.
Adjust golden output accordingly.
Fixes PR 53150.
2018-04-09 16:21:05 +00:00
kamil 8e4f836c68 Add new ATF tests for Address Sanitzier (ASan)
Add new C and C++ tests:
 - t_asan_double_free
 - t_asan_global_buffer_overflow
 - t_asan_heap_overflow
 - t_asan_off_by_one
 - t_asan_uaf

Each tests checks:
 - regular build
 - 32-bit
 - PIC
 - PIE
 - profile

These tests require paxctl(8) to disable ASLR in order to work in a
predictable way. This is especially true for all !regular builds with
additional compiler flags.

There are no static variations of these tests as this mode is not supported
in upstream ASan.

Enable these tests on amd64 and i386.

This is part two patch, adding the remaining C++ changes.

Patch submitted by <Siddharth Muralee>
Additional polishing by myself.
2018-04-04 23:53:26 +00:00
kamil 9f0bdbd997 Add new ATF tests for Address Sanitzier (ASan)
Add new C and C++ tests:
 - t_asan_double_free
 - t_asan_global_buffer_overflow
 - t_asan_heap_overflow
 - t_asan_off_by_one
 - t_asan_uaf

Each tests checks:
 - regular build
 - 32-bit
 - PIC
 - PIE
 - profile

These tests require paxctl(8) to disable ASLR in order to work in a
predictable way. This is especially true for all !regular builds with
additional compiler flags.

There are no static variations of these tests as this mode is not supported
in upstream ASan.

Enable these tests on amd64 and i386.

Patch submitted by <Siddharth Muralee>
Additional polishing by myself.
2018-04-04 23:51:35 +00:00
kamil b4fcb5aedb Correct a small regression typo in usr.bin/c++ ATF tests
TESTS_SH= changed to TESTS_SH+=
2018-04-04 15:53:04 +00:00
kamil b269e95d9e Sort entries in ATF tests in tests/usr.bin/c++/Makefile
No functional change intended.
2018-04-04 15:08:59 +00:00
kamil 77dc83fa09 Add new C++ ATF tests
Add new variations for existing C++ tests:
 - Static
 - Profile+32-bit
 - PIC+32-bit
 - PIC+Profile
 - PIC+Profile+32-bit

All tests pass for NetBSD/amd64:
 - t_cxxruntime
 - t_hello
 - t_static_destructor

+--------------------------------------------------------------+
| Options            | cxxruntime | hello  | static_destructor |
+--------------------------------------------------------------+
| None               | Passed     | Passed | Passed            |
| 32-bit             | Passed     | Passed | Passed            |
| PIC                | Passed     | Passed | Passed            |
| PIE                | Passed     | Passed | Passed            |
| Profile            | Passed     | Passed | Passed            |
| Static             | Passed     | Passed | Passed            |
| Profile+32-bit     | Passed     | Passed | Passed            |
| PIC+32-bit         | Passed     | Passed | Passed            |
| PIC+Profile        | Passed     | Passed | Passed            |
| PIC+Profile+32-bit | Passed     | Passed | Passed            |
+--------------------------------------------------------------+

Add new C++11 std::call_once tests:
 - t_call_once
 - t_call_once2

Add new C++11 test with pthread_once(3) and C++ lambda:
 - t_pthread_once

All tests with the profile option for std::call_once and
pthread_once(3) are marked as expected failure (NetBSD/amd64).

Results for *_once*:
+------------------------------------------------------------+
| Options            | call_once | call_once2 | pthread_once |
+------------------------------------------------------------+
| None               | Passed    | Passed     | Passed       |
| 32-bit             | Passed    |  Passed    | Passed       |
| PIC                | Passed    | Passed     | Passed       |
| PIE                | Passed    | Passed     | Passed       |
| Profile            |  Failed   |  Failed    |  Failed      |
| Static             | Passed    | Passed     | Passed       |
| Profile+32-bit     |  Failed   |  Failed    |  Failed      |
| PIC+32-bit         | Passed    | Passed     | Passed       |
| PIC+Profile        |  Failed   |  Failed    |  Failed      |
| PIC+Profile+32-bit |  Failed   |  Failed    |  Failed      |
+------------------------------------------------------------+

Long term there is an option to refacotr the framework for C and C++ tests,
in order to reduce code duplication.

Patches sent by Yang Zheng <tomsun.0.7@gmail.com>
2018-03-24 00:26:51 +00:00
kre 79eedfcd84 Do the previous test a better way - for a file, test is generally
adequate, but for a device, we really need to actually try opening
it to determine that it is possible - so do the test that way, then
if the open succeeds once, assume it will the second time (which then
holds it open.)
2017-07-25 22:28:22 +00:00
kre ddab1f6e3b Correct oversight in previous ... redirecting into a compound statement
causes the shell to exit if the redirect fails (posix says "may exit"
and /bin/sh does - maybe should give that more thought) - which will
happen if /dev/pad0 does not exist, causing a very messy test abort
(the shell running the test is not supposed to just go away).  So
check tha the device exista and is readable before attempting to open it.

Problem brought to my attention by nat@ - thanks...
2017-07-25 21:25:03 +00:00
kre ae98bc7d08 NFC: Typo in a comment corrected. 2017-07-18 13:17:37 +00:00
kre a94bbc243b Make sure that the open of /dev/pad0 has succeeded (or at the very
least been attempted) before attempting to open /dev/mixer to determine
if the system being tested has audio or not.  Leaving this for the background
cat command to do causes a race between that command and the parent sh.
Move this code to a helper function to avoid duplicating it.

Also avoid attempting to kill the background cat if it was never created.
The kill is likely unnecessary anyway, ATF seems to clean up processes
like that one without assistance.   Which is a good thing, as the kill
does not happen if the test is skipped, or fails.
2017-07-18 12:40:57 +00:00
nat 27ecfe9c27 As pad devices are now created on demand - pad has to be open for a
corresponding mixer to be available.
2017-07-03 09:08:35 +00:00
martin b40441be27 In the -m32 test, additionally test profiled binaries. 2017-05-31 11:08:35 +00:00
martin 4fcb013cbf Add (currently failing) test cases for profiled programs. 2017-05-18 10:29:47 +00:00
kamil d1be05c0a9 Add new ATF C++ tests in usr.bin/c++: t_static_destructor
These tests are cloned from t_cxxruntime and check proper order of destructor
calls. They must be reported in reverse order of constructor completion.

Added tests:
 - static_destructor
 - static_destructor_pic
 - static_destructor_pie
 - static_destructor32

This test file replaces src/regress/usr.bin/c++/static_destructor.
2017-05-14 02:02:25 +00:00
kamil 179f19cf82 Use more C++ sugar in t_cxxruntime (ATF test) 2017-05-14 01:13:44 +00:00
kamil 9fab767d88 Add new ATF C++ tests in usr.bin/c++
These tests are cloned from t_hello and use c++ runtime basic functions.

Added tests:
 - hello
 - hello_pic
 - hello_pie
 - hello32
2017-05-14 01:03:21 +00:00