that we can always wait(2) for our children, and an ignored SIGCHLD
prevents that. Recent versions of bash can be convinced (due to a
bug most likely) to invoke us that way. Always return SIGCHLD to
SIG_DFL during init - we already prevent scripts from fiddling it.
All ash derived shells apparently have this problem (observed by
Martijn Dekker, and notified on the bash-bug list). Actual issue
diagnosed by Harald van Dijk (same list).
Mark the variable as volatile as it can be clobbered when a vfork occurs.
Error was reported when build.sh was run with MKLIBCSANITIZER=yes flag.
Reviewed by: kamil@
https://austingroupbugs.net/view.php?id=252
the Austin Group decided to require processing of "--" by the "."
and "exec" commands to solve a problem where some shells did
option processing for those commands (permitted) and others did
not (also permitted) which left no safe way to process a file
with a name beginning with "-".
This has finally made its way into what will be the next version of
the POSIX standard.
Since this shell did no option processing at all for those commands,
we need to update. This is that update.
The sole effect is that a "--" 'option' (to "." or "exec") is ignored.
This means that if you want to use "--" as the arg to one of those
commands, it needs to be given twice ". -- --". Apart from that there
should be no difference at all (though the "--" can now be used in other
situations, where we did not require it before, and still do not).
process with redirects. If we use vfork() and a redirect hangs
(eg: opening a fifo) which the parent was intended to unhang,
then the parent never gets to continue to unhang the child.
eg: mkfifo f; cat <f &; echo foo>f
The parent should not be waiting for a background process, even
just for its exec() to complete. if there are no redirects there
is (should be) nothing left that might be done that will cause any
noticeable delay, so vfork() should be safe in all other cases.
If a builtin command or function is the final command intended to be
executed, and is interrupted by a caught signal, the trap handler for
that signal was not executed - the shell simply exited (an exit trap
handler would still have been run - if there was one the handler
for the signal may have been invoked during the execution of the
exit trap handler, which, if it happened, is incorrect sequencing).
Now, if we're exiting, and there are pending signals, run their handlers
just before running the EXIT trap handler, if any.
There are almost certainly plenty more issues with traps that need
solving. Later,
XXX pullup -9
(-8 is too different in this area, and this problem suitably obscure,
that we won't bother) (the -7 sh is simply obsolete).
Having traps set should not enforce a fork for the next command,
whatever that command happens to be, only for commands which would
normally fork if they weren't the last command expected to be
executed (ie: builtins and functions shouldn't be exexuted in a
sub-shell merely because a trap is set).
As it was (for example)
trap 'whatever' SIGANY; wait $anypid
was guaranteed to fail the wait, as the subshell it was executed
in could not have any children.
XXX pullup -9
GCC_NO_FORMAT_TRUNCATION -Wno-format-truncation (GCC 7/8)
GCC_NO_STRINGOP_TRUNCATION -Wno-stringop-truncation (GCC 8)
GCC_NO_STRINGOP_OVERFLOW -Wno-stringop-overflow (GCC 8)
GCC_NO_CAST_FUNCTION_TYPE -Wno-cast-function-type (GCC 8)
use these to turn off warnings for most GCC-8 complaints. many
of these are false positives, most of the real bugs are already
commited, or are yet to come.
we plan to introduce versions of (some?) of these that use the
"-Wno-error=" form, which still displays the warnings but does
not make it an error, and all of the above will be re-considered
as either being "fix me" (warning still displayed) or "warning
is wrong."
a bracket expression in a pattern (ie: [[:THISNAME:]]). Previously
the code used strspn() to look for invalid chars in the name, and
then memcpy(), now we do the test and copy a character at a time.
This might, or might not, be faster, but it now correctly handles
\ quoted characters in the name (' and " quoting were already
dealt with, \ was too in an earlier version, but when the \ handling
changes were made, this piece of code broke).
Not exactly a vital bug fix (who writes [[:\alpha:]] or similar?)
but it should work correctly regardless of how obscure the usage is.
Problem noted by Harald van Dijk
XXX pullup -9
our implementation was fine, but the restrict marker is problematic
as gcc 8 is now more strict about checking for restrict issues.
this is the only actual consumer of swab(3) in our tree, though,
besides the test for it. oh well.
The new member is caled f_mntfromlabel and it is the dkw_wname
of the corresponding wedge. This is now used by df -W to display
the mountpoint name as NAME=
like the other tty ioctls that we only warn about). Do the main
ioctl (tcgetattr) first, since that provides a better error message
(ENOTTY instead of EINVAL).
The RSS related statistics are now back in the NetBSD kernel.
These values were disabled since day0 until today.
libkvm(3) users will still receive inappropriate values as RSS statistics
are updated upon sysctl(3) call.
Patch submitted by <Krzysztof Lasocki>
(inside a function or dot script) the exit status of that return
statement should become the exit status of the function (or dot
script) - we were ignoring it,
That is
fn() { while return 7; do return 9; done; return 11; }
should exit with status 7. It was exiting 0.
This is apparently another old ash bug that has been fixed
everywhere else in the past.
Issue pointed out by Martijn Dekker, (fairly obvious) fix borrowed
from FreeBSD, due for return sometime next century.
in the past, but managed to re-surface...
The expression "${0+\}}" should expand to "}" not "\}"
Almost all other shells handle it that way (incl FreeBSD & dash).
Issue pointed out by Martijn Dekker.
Add ATF sub-tests for the 4 old var expand operators (${var+word}
${var-word} ${var-word} and ${var?word} - including the forms
with the ':' included) and amongst those tests include test cases
for this issue, so if the bug tries to appear again, we can squash
it quicker. (The newer pattern matching operators are already
well tested as part of testing patterns.)
cleanups to the trap code. No longer silently ignore attempts to
do anything other than set SIGKILL or SIGSTOP to the default ('-")
state. Don't include those in trap or trap -p output (the former
because they cannot be other than in default state, so simply aren't
included, the latter because it is pointless) but do list them
when requested with trap -p SIG.
Interactive mode SIGINT traps are now run ASAP, rather than after
a command has been entered (so the sequence ^C \n is no longer needed
to generate one). Further, when trapped, in interactive mode,
while waiting for a user command, a SIGINT acts (aside from the
trap being run) just like when not trapped, aborts the command being
entered (rather than leaving it, which it did when libedit was in use)
prints a new prompt, and starts again (which is what should happen.)
Traps other than SIGINT (which has always been handled special in
interactive mode) are unaffected by this change, as are SIGINT traps
in non-interactive shells. Or that is the intent anyway.
Fix an in_dotrap ref count bug (was never being decremented... that
was inserted in a place never executed) (relatively harmless) and
add/improve some trap/signal related DEBUG mode tracing.
Update the description of the <& and >& redirection operators
(as indicated would happen in a message appended to the PR a week ago,
which received no opposition - no feedback).
Some rewriting of the section on redirects (including how the word
expansion of the "file" works) to make this simpler & more accurate.
Fix handling of "$@" (that is, double quoted dollar at), when it
appears in a string which will be subject to field splitting.
Eg:
${0+"$@" }
More common usages, like the simple "$@" or ${0+"$@"} end up
being entirely quoted, so no field splitting happens, and the
problem was avoided.
See the PR for more details.
This ends up making a bunch of old hack code (and some that was
relatively new) vanish - for now it is just #if 0'd or commented out.
Cleanups of that stuff will happen later.
That some of the worst $@ hacks are now gone does not mean that processing
of "$@" does not retain a very special place in every hackers heart.
RIP extreme ugliness - long live the merely ordinary ugly.
Added a new bin/sh ATF test case to verify that all this remains fixed.
finding a job that had previously terminated.
Now in that case JOBWANTED is set on all jobs (since any will do)
which then simplifies a later test which no longer needs to special
case "wait -n". Further, we always look to see if any wanted
job has already terminated, even if there are still running jobs
we can wait upon - if anything is already ready, that's where we start
harvesting (and finish, if -n is specified).
Stamp out "greengrocers' apostrophes" in various places (arguably there
are still more present, but style guides vary on that, and my energies
spent corralling wayward punctuation marks could be spent elsewhere).
As a somewhat pedantic clarification, "-s" does not accept backslashes
as delimiters. (While here, also make the macro use of an expression
shared between pax.1 and tar.1 consistent.)
Note the "s" option has an "s" flag that "prevents substitutions from
being performed on symbolic link destinations". Carry over r. 1.25 from
christos@ and part of r. 1.26 from wiz@ from tar.1, since this
functionality is available in pax as well as tar.
integer (previously it was just clamped at the max possible value).
This would have caused
sleep 10000000000000000000
(or anything bigger) to have only actually slept for 9223372036854775807
secs. Someone would have noticed that happen, one day, in some other
universe.
This is now an error, as it was previously if this had been entered as
sleep 1e19
Also detect an attempt to sleep for so long that a time_t will no longer
be able to represent the current time when the sleep is done.
Undo the attempts to work around a broken kernel nanosleep()
implementation (by only ever issuing shortish sleep requests,
and looping). That code was broken (idiot botch of mine) though
you would have had to wait a month to observe it happen. I was going
to just fix it, but sanity prevailed, and the kernel got fixed instead.
That allows this to be much simplified, only looping as needed to
handle dealing with SIGINFO. Switch to using clock_nanosleep()
to implement the delay, as while our nanosleep() uses CLOCK_MONOTONIC
the standards say it should use CLOCK_REALTIME, and if that we
ever changed that, the old way would alter "sleep 5" from
"sleep for 5 seconds" to "sleep until now + 5 secs", which is
subtly different.
Always use %g format to print the original sleep duration in reports of how
much time remains - this works best for both long and short durations.
A couple of other minor (frill) mods to the SIGINFO report message as well.
with an IQ that underflows when one attempts to enter it as an
unnormalised 160 bit long long double...
Whoever would believe that (~0 & anything) was a meaningful thing
to write? And three times in one #define. That could not possibly
have been me, could it?
Simplify, simplify, simplify. NFC.
to allow bash to build fdflags on Solaris 10, here are some mods that
fix that, and some other similar issues in the NetBSD version of fdflags.
The bash implementation of fdflags is based upon the one Christos did for
the NetBSD sh, so the issues are similar ... the NetBSD sh cannot yet
(easily anyway) build on anything except NetBSD, so this change makes
no current difference at all (just adds some compile time tests (#ifdef)
which always work out the way things did before, when built on NetBSD).
However, there is no system on which any modern shell can hope to work
which does not support close on exec, or fcntl(F_SETFD,...) to set it.
The O_CLOEXEC and FD_CLOEXEC definitions might not exist, but close on
exec can still be manipulated. Since the primary rationale for
the fdflags builtin was to be able to manipulate that state bit from
scripts, it would be annoying to lose that one, and keep all the (less
important) others, just because O_CLOEXEC is not defined, so do the
fix (workaround) a different way than was done in the bash patch.
Further, more than fdflags() will fail if O_CLOEXEC is not defined,
so handle that as well.
Also fix another oddity ... (noticed by reading the code) - if
fcntl(F_GETFL,...) returned any bits set that we don't understand,
the code was supposed to simply print their values as a hex constant,
when fdflags is run with -v. However, the getflags() function was
clearing all bits that the code did not know about ... so there is
no way any unknown bit could ever make it out to be printed. Handle
that a different way - instead of clearing unknown bits, clear any
bits that get returned which we understand, but do not want to deal
with (stuff like O_WRONLY, which should not be returned from the
fcntl(), but who knows...) Leave any unknown bits that happen to be
set, set, so that printone() can display them if appropriate.
(This is most likely to happen when running an older shell on a new
kernel where the kernel supports some new flag that the shell has
not been taught to understand).
NFCI that anyone should notice anytime soon.
matches the internal CTL* chars.
The earlier fixes handled CTL* char values in var expansions,
but not in various other places they can occur (positional
parameters, $@ $* -- even potentially $0 and ~ expansions,
as well as byte strings generated from a \u in a $'' string).
These should all be correctly handled now. There is a new
ISCTL() macro to make the test, rather than using the old
BASESYNTAX[c]==CCTL form (which us still a viable alternative)
as the new way allows compiler optimisations, and less mem
references, so it should be smaller and faster.
Also, be sure in all cases to remove any CTLESC (or other)
CTL* chars from all strings before they are made available
for any external use (there was one case missed - which didn't
matter when we weren't bothering to escape the CTL* chars at
all.)
XXX pullup-8 (will need to be via a patch) along with the Feb 4 fixes.
tree, don't display a CTLESC which is there only to protect a CTL*
char (a data char that happens to have the same value). No actual
CTL* chars are printed as data, so no escaping is needed to protect
data which just happens to look the same. Dropping this avoids the
possibility of confusion/ambiguity in what the word actually contains.
NFC for any normal shell build (very little of this file gets compiled there)
anway) on tech-userlevel with no adverse response.
This allows the magic of vars like HOSTNAME SECONDS, ToD (etc) to be
restored should it be lost - perhaps by having a var of the same name
imported from the environment (which needs to remove the magic in case
a set of scripts are using the env to pass data, and the var name chosen
happens to be one of our magic ones).
No change to SMALL shells (or smaller) - none of the magic vars (except
LINENO, which is exempt from all of this) exist in those, hence such a
shell has no need for this command either.
redirect operator is within range of what the code tree node can
hold. Currently this is a no-op change (the new error can never
occur) as the code already checks that N is in range for an int
(and errors if not) and the field in the node in which we store N
is also an int, so we cannot overflow - but fd's do not really need
to be that big (the max a typical kernel supports is < 10000) so
this just adds validation in case it ever happens that we decide we
can save some node size (ie: sh memory) by making that field smaller.
Note this is parse time error detection, and has no bearing upon
the execution time error that will occur if a script attempts to use
an fd that exceeds the process's max fd limit.
NFCI (for now anyway.)
a value. There are none which do that at the minute, so this is a NFCI
change, which is just making the code correct even though nothing
currently triggers any bugs.
%x commands) generate the most useful error message (from errno value)
rather than whichever happened last.
In posix mode, cause the "jobs" command to delete records of completed
jobs it reports on (as posix requires) as is done in interactive shells.
We don't (won't) do this in !posix mode, as the ability to throw in a
"jobs" command in a script to debug what is happening is too useful to
lose -- and any script that is relying on "jobs" instead of "wait" to
cleanup background processes (from the sh jobs table, sh always collects
zombies from the kernel) is absurd and not worth considering (besides
which I've never seen one).
No visible differences expected - there is a remote chance that
some internal lossage may no longer occur in interactive shells
that receive SIGINT (untrapped) at inopportune times, but you would
have had to have been very unlucky to have ever suffered from that.
Suppress shell error messages while expanding $ENV (which also causes
errors while expanding $PS1 $PS2 and $PS4 to be suppressed as well).
This allows any random garbage that happens to be in ENV to not
cause noise when the shell starts (which is effectively all it did).
On a parse error (for any of those vars) we also use "" as the result,
which will be a null prompt, and avoid attempting to open any file for ENV.
This does not in any way change what happens for a correctly parsed command
substitution (either when it is executed when permitted for one of the
prompts, or when it is not (which is always for ENV)) and commands run
from those can still produce error output (but shell errors remain suppressed).
fixes) where a variable containing a CTL char (the only possibility used
to be CTLESC (0x81)) would lose that character if the variable was expanded
when "set -f" (noglob) was in effect.
1.128 made this worse by adding more 0x8z values (a couple more) which would
see the same behaviour, and one of those was noticed by Martijn Dekker.
The reasoning was that when noglob is on, when a var is expanded, there are
no magic chars, so (apparently) no need to escape anything. Hence nothing
was escaped .. including any CTL chars that happened to be present. When
we later rmescapes() the CTL chars that we expect might occur are summarily
removed - even if they weren't really CTL chars, but just data masquerading.
We must *always* escape any CTL char clones that are in the var value,
no matter what other conditions apply, and what we expect to happen next.
While here, fix rmescapes() (and its $(()) clone, rmescapes_nl()) to
be more robust, less likely to forget to delete anything (which was
not the issue here, just the reverse) and in a DEBUG shell, have the
shell abort() if it encounters something in rmescapes() it is not
anticipating, so the code can be made to handle it, or if it should
not happen, we can find out why it did.
XXX pullup -8 (but will need to be via patch, code is quite different).
After all, a system might want to sleep for several
thousand years on a spaceship headed to a distant
solar system...
So, remove the pause() code, deal with limits on the
range (it is just an int) that can be passed to sleep()
by looping, and do a much better job of checking for
out of range input values.
With this change sleep(1) should work for durations
up to something more than 250 billion years. It
fails (at startup, with an error) if the requested
duration is beyond what can be handled.
Here no changes at all related to locales and arg
parsing. Still for another day.
integer" case, so we avoid adjusting the locale of sleep,
and generally be more reliable and simpler.
In addition, deal with weirdness in nanosleep() when the
interval gets long, by avoiding using it. In this version
when the sleep interval < 10000 seconds, we use nanosleep()
as before, for delays longer than that we use sleep() instead,
and ignore any fractional seconds.
We avoid overflow problems here by not bothering to sleep
at all for delays longer than 135 years (approx) and simply
pause() instead. That sleep never terminates in such a
case is unlikely to ever be observed.
This commit makes no decision on the question of whether
the arg should be interpreted in the locale of the user,
or always in the C locale. That is for another day.
src/tests/bin/sh/t_here.sh
The "magicq" magic was all wrong - it cannot be simply a parameter
to readtoken1() as its value needs to alter during that routine
(eg: when magicq is set - processing here doc text, or whatever)
and we encountered ${var%pattern} "magicq" needs to be off for
"pattern" - and it wasn't.
To handle this magicq needs to be included in the token stack struct,
and simply init'd from the arg to readtoken1 (which we rename).
Then it can be manipulated as required.
Once we no longer have that problem, some other issues can be cleaned
up as well (some of this unbelievably fragile code was attempting to
cope with this in various ad-hoc - and mostly broken - ways).
Also, remove the magicq parameter from parsebackq() - it was not
used (at all) and should never be, a command substitution, wherever
it appears, always starts a new parsing context. How that applies
to old style command substitutions is less clear, but until we see
some real examples where we're not doing the right thing (slightly
less likely now than before ... nothing has changed here in the
way command substitutions are parsed, but quoting in general is
slightly better) I don't plan on worrying about it.
There are a couple of other minor cleanups, which make no actual
difference (like adding () around the use of the parameter in the
RETURN macro ... which is generally better, but makes no difference
here as the param is always a simple constant.
All the current ATF tests pass.
Add tracing of lexical analyser operations. This is deliberately
kept out of the normal "all on" set as it makes a *lot* of noise
when enabled (especially in verbose mode) - but when needed, it
helps (evidence for which is coming soon).
As usual, no doc, you need the sources (and of course, a specially
built sh to even be able to enable it.)
Add an error DEBUG trace in exraise() (when the shell has detected
some error or signal, and is aborting what it is doing)
Fix an arith error in DEBUG bit assignments (harmless as we haven't
reached the limit of flags yet), and add some missing (recently added)
debug flags so they are turned on when the user (ie: me) asks for
"everything".
(PS1 etc) which, if the shell were already exiting, and a prompt
were to be expanded (which only really happens if -x is enabled,
and an exit trap is set, so the commands in the trap need PS4
expanded and written, last thing, before the shell exits) the shell
would instead simply exit when it finished expanding PS4 (before
even writing it, or the xtrace output).
There were more conditions required to set up the environment for
this to actually occur (it seems to only happen when the exit trap
is set in a function, called in a command substitution) but that's
unimportant, the code was nonsense.
Problem noticed by Martijn Dekker.
XXX pullup -8
(which means this is the very first execution in a new subshell)
clear the traps completely, unless the command is "trap". We were
allowing any special builtin, which was probably harmless, but not
intended.
Also (though not required) permit "command trap" and "eval trap"
and combinations thereof, because they might be useful, and there is
no particular reason why not. This is all a part of making t=$(trap)
work as POSIX requires, but almost nothing beyond that. The "trap"
command must be alone (modulo eval and command) in the subshell for
the exception to apply, no t=$(trap; echo) or anything like that.
Martijn Dekker asked for "command trap" to work (no idea why though,
it converts "trap" from being a special builtin, to a normal one,
which means an error won't cause the shell to exit ... if there's
an error, the "trap" command won't do anything useful, and as we
permit no more commands (for this special treatment) the shell is
going to exit anyway, this difference is not really significant.
RANDOM initialisation failed
when the shell might print after RANDOM has been reseeded
(which includes at sh startup) the next time RANDOM is accessed.
It indicates that /dev/urandom was not available or did not
provide data - in that case, sh uses a (weak) seed made out of
the pid and time (but otherwise nothing else changes).
one in the "safe" way (it was ensuring the buffer always ended in 2 \0
characters ... one is enough.) This could affect the expansions of
LINENO RANDOM and SECONDS, though only if they have at least 8 digits
(and then, only sometimes). RANDOM thus is safe, as it never produces
a number with more than 5 digits, you'd need a script with 10000000
lines before there might be an issue with LINENO (and even autoconf
generated scripts don't generally get that bit) and a shell would need
to be running for almost 4 months for SECONDS to climb that high.
Nevertheless: XXX pullup -8.
includes typing ^D) make sure LINENO is set to indicate the last
(actually one past last) line in the input file, rather than
whatever it was set to by the last command that was actually
executed (which could be some line in a function defined in
some other file).
No effect on exit via an explicit exit command - that would already
set the line number correctly.
what the current locale's radix character happens to be,
while still allowing locale specific entry of fractional
seconds (ie: if you're in locale where the radix character
is ',' you san use "sleep 2.5" or "sleep 2,5" and they
accomplish the same thing).
This avoids issues with the "sleep 0.05" in rc.subr which
generated usage messages when a locale that does not use
'.' as its radix character was in use.
Reported on netbsd-users by Dima Veselov, with the problem
diagnosed by Martin Husemann
While here, tighten the arg validity checking (3+4 is
no longer permitted as a synonym of 3) and allow 0.0
to mean the same thing as 0 rather than being an error.
Also, make the SIGINFO reports a little nicer (IMO).
The ATF tests for sleep all pass (not that that means a lot).
will necessarily contain. Allow defined nodes to use any
intN_t or unintN_t (as well as plain old int) data types
in fields (along with the others that are permitted).
Note: this script is a part of the build procedure for /bin/sh,
the modified version generates the exact same output files
(for the unaltered input specifications) as the previous one
did, hence no visible change is expected (or even possible).
While there is a tiny chance that some host shell will fail
to be able to run this script while building, the script still
uses nothing even slightly exotic, and is much more conservative
than other scripts used during the build process, so there should
be no issues there either.
that when traps are marked as invalid, we never use them
for anything except output from the trap command.
Fixes issues where sub-shells of shells which use traps
(eg: to trap SIGPIPE) can end up looping forever if the
signal occurs in a sub-shell (where the trap is supposed
to be reset to its default). Reported, and mostly
analyzed by Martijn Dekker.
that is #if 0'd and (still) has never been compiled (most likely
never will be.)
While here, in the same uncompiled code, deal with line number
counting. Whether this is correct depends upon how this code
is used, and as it never is (and never has been since line numbers
first started being counted), this is somewhat speculative, but
it seems likely to be the correct way to handle things.
NFC (this code is still all #if 0).
should be looked up as a potential following alias - if the first
expands to a string that ends with a space (any space, quoted or
not) then the next word is to be treated as an alias candidate.
(POSIX was to specify only unquoted spaces, but is now going to
leave that unspecified, and the "any space" version turns out to
be more useful.
And besides, the quoteflag test didn't work properly, and would
have been very messy to fix ... if in a word (as if we have a
quoted space) it means that the word has been quoted, which meant
that quoted spaces were correctly detected, but it outside a word,
it just means that the previous word was quoted, so it would sometimes
reject alias lookup on the next word in cases where it is unquestioned
it should be done.
so the fake char returned by the latter when an alias ends (which
is there so we can correctly avoid alias recursion) is correctly
ignored where it is not wanted.
aliases, to actually do what it was supposed to do, and not just
come close by accident. (How broken this was, while still seeming
to work perfectly most of the time was truly amazing!)
This corrects the behaviour of an alias defined with a blank char
as the last of its value, to correctly do an alias lookup on the
word that follows the alias.
