New tests attempting to kill, stop, drop or revive a zombie:
- signal1 (SIGKILL)
- signal2 (SIGSTOP)
- signal3 (SIGABRT)
- signal4 (SIGHUP)
- signal5 (SIGCONT)
New test race1 verifying whether there are any kernel races when processing
signals to zombies, executing in a loop for 5 seconds.
These tests were inspired by a kernel unexpected behavior when a lookup
of a dying process could result in two detected entities once as an alive
process and once as a zombie.
race1 is similar to t_ptrace_wait* race1, however without ptrace(2) involved.
Sponsored by <The NetBSD Foundation>
Reuse the attach1's test body for race1.
Add a new test race1:
Assert that await_zombie() in attach1 always finds a single
process and no other error is reported
race1 requires HAVE_PID in wait(2)-like function.
This test is executed in a loop for 5 seconds (16k iterations on Intel i7).
A buggy kernel was asserting an error within this timeframe almost always.
The bug in the kernel is now gone and this test is expected to pass
correctly.
Sponsored by <The NetBSD Foundation>
Add await_zombie_raw() that is the same as await_zombie(), whith an
addition of additional "useconds_t ms" parameter indicating delays between
new polling for a zombie process.
This new function will be used for testing a race condition that has been
observed occassionally crashing a test case -- returning duplicate entries
for KERN_PROC_PID.
Sponsored by <The NetBSD Foundation>
The primary race specific to this test has been fixed in previous commit
(wrong WNOHANG).
This test is still racy and breaks like once every 30,000 execution.
This is down like from once from every 100th execution in the past.
The remaning race is not specific to attach2 and I can reproduce it with
at least attach1. It still looks like being specific to NetBSD and it's
not reproducible on Linux and FreeBSD. Perhaps a bug with pipe(2)/write(2)/
read(2) or close to these features.
Sponsored by <The NetBSD Foundation>
At the end of the test we resume a tracer and expect to observe it to
collect the debuggee. We cannot from a parent point of view wait for
collecting it with WNOHANG without a race.
Remove the WNOHANG option from wait*(2) call. This corrects one type of
race.
This test is still racy for some other and unknown reason and this is being
investigated.
Sponsored by <The NetBSD Foundation>
This code after refactoring stopped calling functions that were designed
to trigger expected behavior and thus, tests were breaking.
Sponsored by <The NetBSD Foundation>
These operations cloned Linux's specific PTRACE_GETSIGMASK / PTRACE_SETSIGMASK.
This feature was useful in applications like rr/criu/reptyr-like, where
the ptrace(2) interface is abused for the purpose of constructing an arbitrary
process. It's not reliable and not portable. For the NetBSD case it will be
better to invent something dedicated for serializing and deserializing a
process with threads.
Noted on tech-toolchain@ and blog entry
"LLDB restoration and return to ptrace(2)"
https://blog.netbsd.org/tnf/entry/lldb_restoration_and_return_to
The <inttypes.h> user-land header is required for PRI types.
We were including indirectly <sys/inttypes.h> through <sys/*.h> sources,
and this worked for most of the ports.
This fixes indirectly a build for MIPS (tested ports: arc and hpcmips),
where PRI types (PRIxREGISTER) were undefined.
Noted by <uwe>.
Cast _UC_MACHINE_*(&u) to (register_t).
This allows to print the returned values with PRIxREGISTER.
This corrects build failures on MIPS and HPPA.
sizeof(int) == sizeof(long) on these 32-bit systems.
New tests:
- ucontext_sp
- ucontext_fp
- ucontext_pc
- ucontext_intrv
They test respectively:
- _UC_MACHINE_SP
- _UC_MACHINE_FP
- _UC_MACHINE_PC
- _UC_MACHINE_INTRV
These tests attempt to access and print the values from ucontext, without
interpreting the values.
This is a follow up of the _UC_MACHINE_FP() introduction.
These tests use PRIxREGISTER, and require to be built with -D_KERNTYPES.
Sponsored by <The NetBSD Foundation>
with FreeBSD)
* Fix a broken call to open(2) with O_CREAT and no permission argument.
* ANSIfy function definitions.
* Improve failure messages.
From freebsd.org via Brooks Davis - Thanks!
convert several raidframe ioctls to be bitsize idempotent so that
they work the same in 32 and 64 bit worlds, allowing netbsd32 to
configure and query raid properly. remove useless 'row' in a few
places. add COMPAT_80 and put the old ioctls there.
raidframeio.h:
RAIDFRAME_TEST_ACC
- remove, unused
RAIDFRAME_GET_COMPONENT_LABEL
- convert to label not pointer to label
RAIDFRAME_CHECK_RECON_STATUS_EXT
RAIDFRAME_CHECK_PARITYREWRITE_STATUS_EXT
RAIDFRAME_CHECK_COPYBACK_STATUS_EXT
- convert to progress info not pointer to info
RAIDFRAME_GET_INFO
- version entirely.
raidframevar.h:
- rf_recon_req{} has row, flags and raidPtr removed (they're
not a useful part of this interface.)
- RF_Config_s{} and RF_DeviceConfig_s{} have numRow/rows removed.
- RF_RaidDisk_s{} is re-ordered slightly to fix alignment
padding - the actual data was already OK.
- InstallSpareTable() loses row argument
rf_compat32.c has code for RF_Config_s{} in 32 bit mode, used
by RAIDFRAME_CONFIGURE and RAIDFRAME_GET_INFO32.
rf_compat80.c has code for rf_recon_req{}, RF_RaidDisk_s{} and
RF_DeviceConfig_s{} to handle RAIDFRAME_FAIL_DISK,
RAIDFRAME_GET_COMPONENT_LABEL, RAIDFRAME_CHECK_RECON_STATUS_EXT,
RAIDFRAME_CHECK_PARITYREWRITE_STATUS_EXT,
RAIDFRAME_CHECK_COPYBACK_STATUS_EXT, RAIDFRAME_GET_INFO.
move several of the per-ioctl code blocks into separate functions.
add rf_recon_req_internal{} to replace old usage of global
rf_recon_req{} that had unused void * in the structure, ruining
it's 32/64 bit ABI.
add missing case for RAIDFRAME_GET_INFO50.
adjust raid tests to use the new .conf format, and add a case to
test the old method as well.
raidctl:
deal with lack of 'row' members in a couple of places.
fail request no longer takes row.
handle "START array" sections with just "numCol numSpare", ie
no "numRow" specified. for now, generate old-style configuration
but update raidctl.8 to specify the new style (keeping reference
to the old style.)
note that: RF_ComponentLabel_s::{row,num_rows} and
RF_SingleComponent_s::row are obsolete but not removed yet.
r1.6 changed the code in such a way that it was hitting a BFD assert on
MIPS, causing the builds to fail.
we were hitting the assert at binutils/dist/bfd/elfxx-mips.c:10879
The workqueue worker can take the mutex before the tester tries to take it after
calling workqueue_enqueue. If it happens, the worker calls cv_broadcast before
the tester calls cv_timedwait and the tester will wait until the cv timed out
Take the mutex before calling workqueue_enqueue so that the tester surely calls
cv_timedwait before the worker calls cv_broadcast.
The fix stabilizes the test, t_workqueue/workqueue1.
will overlap with the requested scheduler range, so get the new scheduler
range, and then try to find a different priority. If that fails (to find
a different scheduling range), give up here.
to my message on tech-userlevel ...
Subject: tests/lib/libpthread/t_mutex:mutex6
Date: Thu, 23 Nov 2017 17:34:54 +0700
Message-ID: <28385.1511433294@andromeda.noi.kre.to>
which can be found at:
http://mail-index.netbsd.org/tech-userlevel/2017/11/23/msg011010.html
which analysed the mutex6 test case of this test, and concluded
that it was useless, nonsense, and broken (the whole test is just a
race - not even really using or testing mutexes), let it be henceforth
forever gone.
more precision bits than the data type expects, but (kind of obviously)
does not allow such values to be stored in memory, expecting the value
returned from strtod() (an intermediate result) to be identical (that is,
equal) to a stored value is incorrect.
So instead go back to checking that the two numbers are very very close.
See comments added to the test for more explanation.
of what are left are "race for the bus" type - if we lose, we just
wait for the next one ... slower but still reliable.
There are two exceptions ... when starting more than one rtadvd
(on different routers) we expect to receive an RA from each, but
all that we can check is that we received the (at least) right number
of RAs. It is possible (though unlikely) that one router sent two
before another sent any, in which case we will not have the data we
expect, and a sub-test will fail.
Second, there is no way to know for sure that we have waited long
enough when we're waiting for data to expire - in systems with
correctly working clocks that actually measure time, this should not
be an issue, if data is due to expire in < 5 seconds, and we wait
5 seconds, and the data is still there, then that indicates a
failure, which should be detected. Unfortunately with QEMU testing
time just isn't that reliable. But fortunately, it is generally the
sleep which takes longer, while other timers run correctly, which is
the way that makes us happy...
While here lots of cleanups - everything from white space and
line wrapping, to removing superfluous quotes and adding some
(but probably not enough) that are not (though given the data is
all known here, lack of quotes will rarely hurt.)
Also take note of the fact that current rtadvd *cannot* delete its
pidfile, so waiting for that file to be removed is doomed to failure.
Do things in a way that works, rather than simply resorting to assassination.
Because we do a lot less "sleep and hope it is long enough" and more
"wait until it is observed to happen" the tests generally run in less
elapsed time than before (20% less has been observed.) But because we
"wait until it is observed to happen" rather than just "sleep and hope
it is long enough" sometimes things take longer (and when that happens,
we no longer fail). Up to 7% slower (overall) has been observed.
(Observations on an amd64 DomU, no idea yet as to what QEMU might observe.)
computed using different methods, don't expect to achieve identical
results (here, one constant is perhaps converted to binary from a string by
a cross compiler, the other is converted at run time). Allow them to
have a small difference (for now, small is < 1e-7 - the constant is ~ 1e5,
so this is 12 orders of magnitude less) before failing (and include the
actual difference in the error message if it does fail.)
1. get pid of bg process with $! not $?
2. expect a single message from route monitor, not two, after ndp -d
3. run atf_check just once to verify correct output, not once for each string
1. get rid of the "$*" fetish.
2. more consistency (not complete .. yet) with RUMP_SERVER setting
3. white space (esp around pipe ('|') symbols.)
4. drop unnecessary \ line joining.
1. Be assertive when claiming the pid of the background route monitor command,
not polite... (ie: $! will give you the pid, $? is just 0 there).
2. Since "wait 0" simply (always) exits with status 127, immediately (we
know without thinking that we have no child with pid 0) the waits were
ineffective - now (after fix#1) they work .. which requires the
route monitor that watches the arp -d to exit after 1 message, not 2,
as 1 is all it gets. (If there really should be 2, someone needs to
find out why the kernel is sending only 1 - I am not that someone).
3. The file contents need to be read only once, no matter how many patterns
we need to look for, save some work, and do it that way (this is not
really a bug,m but saving time for the ATF tests is always a good thing.)
Not sure if this will stop it randomly failing on bablyon5, but it might.
(The likely cause is that the "route.monitor" has not flushed its stdout
buffers at the time the "grep -A 3" [aside: why that way to read the file??]
is performed, so fails to find its expected output ... the route monitor would
get an extra message once interfaces start being destroyed, I assume, and
would exit then, flushing its buffer, but by then it is too late.
If that is/was the cause, then it should be fixed now.)
detected as invalid, become the "someone" referred to in the
previous commit log, and add tests for 0 and 4095 as well, and
while here, throw in a few more that might elicit bugs.
And if the shell running the tests is able, add tests of a few
random vlan tags between 2 and 4093 (1 and 4094 are always tested)
to check that anything in range works (well, partially check...)
Move libevent from being a test playing sub-directory, to a groupy,
just hanging around, hoping someone will notice it, and throw it
a bone... (mixed metaphors?)
Currently (or when testing any shell that does not support -X) the
test will be skipped (also for [m]ksh (but not ksh93 etc) where there
is an absurdly badly named -X option, skip the new test for them as well.)
When -X appears in /bin/sh, this will verify that it is probably working
(the test is MUCH more gruelling than any rational use of -X would be.)
failure occurs, distinguish in the failure message the cases where the
open succeeded (and should not have), and where it failed, but not for
the expected reason (the "wrong errno" case not tested, would require
manufacturing a broken kernel...).
Add O_NOFOLLOW ('l' mode modifier) tests to the fopen_mode test case
(tests that are intended to open the file) and also add a few missing
commas which were causing a few of the sub-tests to be skipped, and
others to be not quite testing exactly what was intended.
to leave errno unaltered if there is no error. And does not.
While here, write -1 the same way everyone else does (not ~0, which
would not even be negative on a 1's complement host, if you can find one).
And while not needed for the test, but so that if checked, the result is
more likely to be what is anticipated, set tm_mday to an in-range value
in the mtime_negyear test (otherwise the correction results in the result
movng backwards to the last day of the previous month, which is the
end of Decemper, 1898, rather than the 1899 one would expect from year -1.)
to its previous state, so any signals that occur during the rest of the
test gat handled just as they would (core dump) as if the handler was never
executed. No need to bother if the SYSVMSG test fails, as in that case,
nothing else will be done anyway.
Currently testing ~user is too much effort to contemplate (other than
assuming that it works in order to verify that it works...) so only bare ~
is being tested for now. Maybe someday...
Right now (@ time of commit), this new test is expected to fail, as ~
expansions are horribly badly broken (have been for months, some forms
for much longer) in all but the simplest of uses. Fix for that coming
very soon.
PROT_MPROTECT(), or if the test is built without _NETBSD_SOURCE defined,
by providing a dummy #define if the real one does not exist. The test
might then fail, if the PROT_MPROTECT() is really needed, but that's
better than not building.
PROT_MAXPROTECT or PROT_ALLOWPROTECT or something) on the mmap() call
which specifies PROT_NONE, and which we later want to change to PROT_READ,
otherwise when PAX is enabled, the mprotect() will fail.
2822 specifications. Unfortunately they are specified incorrectly in
RFC-822 and not very clearly in RFC 2822. RFC 1123 clearly states they
are specified incorrectly - counting the wrong way from UTC - in RFC
822. RFC 2822 just states they were implemented in a non-standard way.
Mea culpa for not noticing when originally implemented. Fix them so
the correct calculations are made.
This test will be skipped on shells (such as /bin/sh in -current as of
the date of this commit) which do not support $'...'
While here fix a typo in a comment (there are probably more...)
The source address or output interface can be specified by adding IP_PKTINFO
to the control part of the message on a SOCK_DGRAM or SOCK_RAW socket.
Reviewed by ozaki-r@ and christos@. thanks.
- Remove unnecessary \ at EOL
- This allows to omit ; too
- Remove unnecessary quotes for arguments of atf_set
- Don't expand $DEBUG in eval
- We expect it's expanded on execution
Suggested by kre@
Add two new sh syntax test cases to check for bug fixes for the parser
problems (syntax errors unidentified) reported in the two PRs.
In the latter case, also include some tests that test similar
looking, valid, syntax - to make sure the fix for the PR does not
break code that should not fail. This is not needed for the earlier
PR as other tests, and just normal parsing, is sufficient.
These tests have been verified to fail with a current /bin/sh and to
pass with the sh updates that are to be committed soon -- and because that
fix is expected within hours, the tests are not marked as expected to
fail, just let them actually fail for now.
adequate, but for a device, we really need to actually try opening
it to determine that it is possible - so do the test that way, then
if the open succeeds once, assume it will the second time (which then
holds it open.)
causes the shell to exit if the redirect fails (posix says "may exit"
and /bin/sh does - maybe should give that more thought) - which will
happen if /dev/pad0 does not exist, causing a very messy test abort
(the shell running the test is not supposed to just go away). So
check tha the device exista and is readable before attempting to open it.
Problem brought to my attention by nat@ - thanks...
Move Unicode <-> ku/ten mapping into the individual codec modules.
Mapping is based on existing iconv data for single-byte encodings,
and included for several, but not all, multibyte encodings.
least been attempted) before attempting to open /dev/mixer to determine
if the system being tested has audio or not. Leaving this for the background
cat command to do causes a race between that command and the parent sh.
Move this code to a helper function to avoid duplicating it.
Also avoid attempting to kill the background cat if it was never created.
The kill is likely unnecessary anyway, ATF seems to clean up processes
like that one without assistance. Which is a good thing, as the kill
does not happen if the test is skipped, or fails.
after than in ksh and bash, and the similar thing in zsh) for which a
request has been made for support in the NetBSD sh.
Until the support is committed, the test will be skipped.
functions are used for destructors of thread_local objects.
If a pending destructor exists, prevent unloading of shared objects.
Introduce __dl_cxa_refcount interface for this purpose. When the last
reference is gone and the object has been dlclose'd before, the
unloading is finalized.
Ideally, __cxa_thread_atexit_impl wouldn't exist, but libstdc++ insists
on providing __cxa_thread_atexit as direct wrapper without further
patching.
complementary writable sysctl for the initial guard size of threads
created via pthread_create. Let the existing attribut accessors do the
right thing. Raise the default guard size for threads to 64KB.
The kernel can have multiple ARP/NDP entries which have an indentical
destination on different interfaces. This is normal and can be
reproduce easily by ping -I or ping6 -S. We should be able to remove
such entries.
arp -d <ip> and ndp -d <ip> are changed to fetch all ARP/NDP entries
and remove matched entries. So we can remove multiple entries
described above. This fetch all and selective removal behavior is
the same as arp <ip> and ndp <ip>; they also do fetch all entries
and show only matched entries.
Related to PR 51179
STACKSTRNUL() bug to eventually be found. Not any of the test cases
directly - the shell running the tests (the same /bin/sh) managed to
build one of the f_variable_syntax sub-tests incorrectly, and that was
enough to eventually allow the bug to be located and squashed.
Like all good promotions, this one comes with increased work, and no extra pay.
We currently parse var expansions like "${x-"a b c"}" incorrectly
according to POSIX (and ancient tradition) though in a way that is
generally unexpcted (there are 2 quoted strings there, according
to the standard, "${x-" and "}" and 'a b c' is unquoted.) This has
never been fixed in our sh, as like in many other shells, it is just
a little unbelievable, most people expect that inside the {} is a
whole new ballpark, and everything starts again, and it does in cases
like "${x%"a b c"}" (the newer matching operators.) There have been
comments in this test explaining how the test is actually testing for
incorrect behaviour for a while now.
But I have since learned of POSIX bug #221 (2010), the resolution of which is
to make the problem case unspecified. This will not actually become part
of posix until the next major version (POSIX 8) whenever that appears (still
years away I expect) - but at least we now can feel happier about the
way our code works and not worry quite so much about testing that incorrect
code keeps working incorrectly....
to 1000 to make the ATF test suite as a whole take less time. Before
the change, this single test case could take more than two hours to
run on a qemu emulated ARM.
It was created to copy FreeBSD, however actually the cache isn't
necessary. Remove it to simplify the code and reduce the cost to
maintain it (e.g., keep a consistency with a corresponding local
route).
produces output with a very large number of consecutive embegged \n
characters.
This test is currently expected to fail (as of commit date) but is not
marked as atf_expect_fail as the shell should be fixed to avoid the
problem quite soon. Until then almost anything might happen to the
sh that runs this test (from just plain wrong results, to core dumps,
even possibly the right results, though that's unlikely).
Whie doing this, get rid of the duplicate implementation of the
nested_cmdsubs_in_heredoc test (which was achieving nothing). I know
how I managed to do that, but on advice of counsel, I ... (it was
just a harmless waste of a tiny amount of CPU time "compiling" the test,
just like writing "x=0;" on consecutive lines....)
These should detect if the errors that caused MAKDEV to fail and
pkgsrc/pkgtools/cwrappers to fail to build (problem detected in
libnbcompat/configure) ever return.
Also fixed one of the other sub-tests so that it actually does what
it should - no idea how this one has been passing, it did not for me
when I was checking the new ones (but perhaps in the interim I have
fixed something else in sh, the problem was in the area I have been
playing, and I originally debugged the new tests using a newer version
of /bin/sh than has yet been committed.)
by IFS just the same as any other expansion (split when they should be,
and not when they shouldn't).
Good thing I did ... this discovered a regression in the new expand
code (from an hour or three ago) where quoted arith expansions are
being split, and shouldn't be. (on the other hand, when they should
be split, they are being split correctly, so that's good...)
No need to worry too much about this, in real life (as opposed to
torture tests) no-one ever puts digits in IFS, and a $(( )) expansion
only ever contains digits, so in practice, splitting never happens,
whether because it is quoted, or just becaue there is nothing to split.
The FreeBSD shell does it correctly (they do word splitting using a
totally different mechanism than we do) - I will find where I missed
testing for quoted expansions later tonight. Until that happens,
expect a test failure from t_fsplit:split_arith
While here add more comments in the other test casess (one had a comment
already) that our shell parses incorrectly (this is a parsing problem,
not an expansion problem, and the bug has existed forever .. and is
shared by bash).
That is, in an expression like "${var:-word}" the whole thing (including
word) is quoted by the double quotes, and in "${var:-"word"}" the
expansion is quoted, but 'word' is not, the 2nd " (the one before 'w')
ends the opening quote, and the third (before }) turns quoting on
again (it is required that there be an even number of unquoted quotes inside
a ${} expression, so things like "${var:-"word} are simply invalid.)
Another way of saying this, is that if the { is quoted (for which the
only way is using " to get to this kind of expansion at all) the }
must also be quoted (" quoted). There is no quote nesting here.
This also means that in "${var:-'word'}" the single quotes are just
characters, they are quoted by the "" that surround them, just as in
"a 'string' containing quotes", but "${var:-"'word'"}} is valid and
contains a single quoted word.
Note that this is different than the patetrn-match/trim expansions
( ${var%pattern} etc) where any surrounding quotes do not affect the
pattern, and all forms of quoting can be used in it - but it always
parses as a pattern, and is never subject to filename expansion, so
a '*' in it that is to mean "match any string" does not need special
quoting to avoid it expanding to file names, regardless of whether
the ${var%*} is quoted or not, but a * that is to be matched as a
character literally does need to be quoted.
I will probably file a PR about this bug sometime, but as it is
very old, and doesn't every seem to bother anyone (and is shared by
bash) there isn't any big hurry to open yet another verry messy can
of excrement to fix it...
This does mean that the FreeBSD shell "fails" some of our tests.
(The test is wrong, their shell is correct.)
as to why ${011} is ${11} and not ${9} (that is, why we interpret it
that way, the "why could it not be the other way?" is just "because
that is not how it was ever implemented".
piece of mind (to verify I was not breaking anything here.)
Also added some commentary better explaining why one of the tests of splitting
quoted variable expansions is almost certainly incorrect (both the tests,
and what sh does) ... though bash does the same as us, so all is not lost!
whose meaning is defined entirely by context.
For those who read commit messages, and want a (small) challenge,
work out where (and what) to insert as punctuation/operator chars
in the following to produce 3 ines of output, and what those will be:
for in in in do in do case in in in echo do do echo in esac done
(There are no comments, quotes of any kind, or any kind of sub-shell,
including cmd substitutions) With correct non alpha-numeric chars added,
it works.
digittoint(3).
The digittoint(3) test is skipped since we don't provide that function yet.
One of the test cases for btowc(3) is also skipped, since it tests conversion
to Unicode---whereas our wchar_t representation is locale-dependent.
roy