Commit Graph

3875 Commits

Author SHA1 Message Date
kre
2ec3f71485 DEBUG mode only change. When pretty-printing a word from a parse
tree, don't display a CTLESC which is there only to protect a CTL*
char (a data char that happens to have the same value).  No actual
CTL* chars are printed as data, so no escaping is needed to protect
data which just happens to look the same.  Dropping this avoids the
possibility of confusion/ambiguity in what the word actually contains.

NFC for any normal shell build (very little of this file gets compiled there)
2019-02-14 13:27:59 +00:00
kre
727a664bee Add the "specialvar" built-in command. Discussed (well, mentioned
anway) on tech-userlevel with no adverse response.

This allows the magic of vars like HOSTNAME SECONDS, ToD (etc) to be
restored should it be lost - perhaps by having a var of the same name
imported from the environment (which needs to remove the magic in case
a set of scripts are using the env to pass data, and the var name chosen
happens to be one of our magic ones).

No change to SMALL shells (or smaller) - none of the magic vars (except
LINENO, which is exempt from all of this) exist in those, hence such a
shell has no need for this command either.
2019-02-14 11:15:24 +00:00
kre
ec83c7c484 Delete a no-longer-used #define that referred to a struct field that
no longer exists.   Also correct a couple of typos in comments.    NFC.
2019-02-13 21:40:50 +00:00
kre
72c6780ab3 Remove a function prototype which was added to <histedit.h> in 2005.
I think we can trust it to be stable by now, and doin't need the dup.
2019-02-10 19:21:52 +00:00
kre
4d2988311a Add a check that the file descriptor mentioned in a N> or N< type
redirect operator is within range of what the code tree node can
hold.   Currently this is a no-op change (the new error can never
occur) as the code already checks that N is in range for an int
(and errors if not) and the field in the node in which we store N
is also an int, so we cannot overflow - but fd's do not really need
to be that big (the max a typical kernel supports is < 10000) so
this just adds validation in case it ever happens that we decide we
can save some node size (ie: sh memory) by making that field smaller.

Note this is parse time error detection, and has no bearing upon
the execution time error that will occur if a script attempts to use
an fd that exceeds the process's max fd limit.

NFCI (for now anyway.)
2019-02-09 09:50:31 +00:00
kre
83735e242c DTRT when dynamically generated variables return "unset" instead of
a value.   There are none which do that at the minute, so this is a NFCI
change, which is just making the code correct even though nothing
currently triggers any bugs.
2019-02-09 09:38:11 +00:00
kre
733a465e66 DEBUG mode change only. Add one extra trace point. NFC for normal builds. 2019-02-09 09:34:43 +00:00
kre
750dbf249a When an interactive shell exits due to an EOF on stdin, send a newline
to stderr (to follow the previous prompt) and cleanup more nicely.
2019-02-09 09:33:20 +00:00
kre
ccf5ffdbe9 In the unlikely event that restarting a job fails (the fg bg and various
%x commands) generate the most useful error message (from errno value)
rather than whichever happened last.

In posix mode, cause the "jobs" command to delete records of completed
jobs it reports on (as posix requires) as is done in interactive shells.
We don't (won't) do this in !posix mode, as the ability to throw in a
"jobs" command in a script to debug what is happening is too useful to
lose -- and any script that is relying on "jobs" instead of "wait" to
cleanup background processes (from the sh jobs table, sh always collects
zombies from the kernel) is absurd and not worth considering (besides
which I've never seen one).
2019-02-09 09:31:33 +00:00
kre
b4a242b5e2 KNF - white space changes, indent using tabs not spaces. NFC. 2019-02-09 09:20:47 +00:00
kre
39879a1c65 DEBUG mode build changes - add extra trace output.
NFC for any normal shell build.
2019-02-09 09:17:59 +00:00
kre
f42ddab0ac Delete extern decl for trap[] - hasn't been needed for a while now. 2019-02-09 09:15:22 +00:00
kre
3dbd860142 Allocate alias pointers for qsort() to use on the stack, rather than
directly via malloc() so they get cleaned up correctly on error/intr.

NFCI.
2019-02-09 09:11:07 +00:00
kre
e8999de45c INTON / INTOFF audit and cleanup.
No visible differences expected - there is a remote chance that
some internal lossage may no longer occur in interactive shells
that receive SIGINT (untrapped) at inopportune times, but you would
have had to have been very unlucky to have ever suffered from that.
2019-02-09 03:35:55 +00:00
wiz
169c18b89d Remove leading zero from date. 2019-02-04 12:18:36 +00:00
kre
4084f829ec PR bin/53919
Suppress shell error messages while expanding $ENV (which also causes
errors while expanding $PS1 $PS2 and $PS4 to be suppressed as well).

This allows any random garbage that happens to be in ENV to not
cause noise when the shell starts (which is effectively all it did).

On a parse error (for any of those vars) we also use "" as the result,
which will be a null prompt, and avoid attempting to open any file for ENV.

This does not in any way change what happens for a correctly parsed command
substitution (either when it is executed when permitted for one of the
prompts, or when it is not (which is always for ENV)) and commands run
from those can still produce error output (but shell errors remain suppressed).
2019-02-04 11:16:41 +00:00
kre
513431119a Add a couple of comments. NFC. 2019-02-04 09:56:48 +00:00
kre
58e34de6bb Fix an old bug (very old) that was made worse in 1.128 (the "${1+$@}"
fixes) where a variable containing a CTL char (the only possibility used
to be CTLESC (0x81)) would lose that character if the variable was expanded
when "set -f" (noglob) was in effect.

1.128 made this worse by adding more 0x8z values (a couple more) which would
see the same behaviour, and one of those was noticed by Martijn Dekker.

The reasoning was that when noglob is on, when a var is expanded, there are
no magic chars, so (apparently) no need to escape anything.  Hence nothing
was escaped .. including any CTL chars that happened to be present.  When
we later rmescapes() the CTL chars that we expect might occur are summarily
removed - even if they weren't really CTL chars, but just data masquerading.

We must *always* escape any CTL char clones that are in the var value,
no matter what other conditions apply, and what we expect to happen next.

While here, fix rmescapes() (and its $(()) clone, rmescapes_nl()) to
be more robust, less likely to forget to delete anything (which was
not the issue here, just the reverse) and in a DEBUG shell, have the
shell abort() if it encounters something in rmescapes() it is not
anticipating, so the code can be made to handle it, or if it should
not happen, we can find out why it did.

XXX pullup -8 (but will need to be via patch, code is quite different).
2019-02-04 09:56:26 +00:00
mrg
f0885992ce - bump buffer sizes to avoid potential truncation issues 2019-02-04 04:36:41 +00:00
mrg
1fcf7be45f - use -Wno-error=implicit-fallthrough with GCC7. 2019-02-04 04:05:15 +00:00
mrg
684b182f81 compare pointers with NULL not '\0'. 2019-02-01 08:29:03 +00:00
wiz
539951b361 Fix typos; use American spelling consistently. Remove an unnecessary macro. 2019-01-30 10:28:50 +00:00
mrg
d9150b46de adjust the open flags available for dd to match actual reality
of what matters.  remove "search" for now, since O_SEARCH has
no backend.  document them all.
2019-01-30 01:40:02 +00:00
wiz
d7fa63f1ef Sort sections. 2019-01-27 17:42:53 +00:00
christos
4b72dcea07 cast to intmax_t instead of long, since time_t is "long long" 2019-01-27 02:00:45 +00:00
martin
4d553ef408 Explicitly cast time_t to match format string - should fix the build on
some 32bit architectures.
2019-01-26 18:14:22 +00:00
kre
a7fe3a0309 While cute, the previous version is not really safe.
After all, a system might want to sleep for several
thousand years on a spaceship headed to a distant
solar system...

So, remove the pause() code, deal with limits on the
range (it is just an int) that can be passed to sleep()
by looping, and do a much better job of checking for
out of range input values.

With this change sleep(1) should work for durations
up to something more than 250 billion years.  It
fails (at startup, with an error) if the requested
duration is beyond what can be handled.

Here no changes at all related to locales and arg
parsing.    Still for another day.
2019-01-26 15:20:50 +00:00
kre
6f62877535 Adjust the way the arg string is parsed in the "not entirely
integer" case, so we avoid adjusting the locale of sleep,
and generally be more reliable and simpler.

In addition, deal with weirdness in nanosleep() when the
interval gets long, by avoiding using it.  In this version
when the sleep interval < 10000 seconds, we use nanosleep()
as before, for delays longer than that we use sleep() instead,
and ignore any fractional seconds.

We avoid overflow problems here by not bothering to sleep
at all for delays longer than 135 years (approx) and simply
pause() instead.   That sleep never terminates in such a
case is unlikely to ever be observed.

This commit makes no decision on the question of whether
the arg should be interpreted in the locale of the user,
or always in the C locale.   That is for another day.
2019-01-26 15:19:08 +00:00
kre
4e25d54034 lexical analysis fixes. This fixes the tests just committed in
src/tests/bin/sh/t_here.sh

The "magicq" magic was all wrong - it cannot be simply a parameter
to readtoken1() as its value needs to alter during that routine
(eg: when magicq is set - processing here doc text, or whatever)
and we encountered ${var%pattern} "magicq" needs to be off for
"pattern" - and it wasn't.

To handle this magicq needs to be included in the token stack struct,
and simply init'd from the arg to readtoken1 (which we rename).
Then it can be manipulated as required.

Once we no longer have that problem, some other issues can be cleaned
up as well (some of this unbelievably fragile code was attempting to
cope with this in various ad-hoc - and mostly broken - ways).

Also, remove the magicq parameter from parsebackq() - it was not
used (at all) and should never be, a command substitution, wherever
it appears, always starts a new parsing context.  How that applies
to old style command substitutions is less clear, but until we see
some real examples where we're not doing the right thing (slightly
less likely now than before ... nothing has changed here in the
way command substitutions are parsed, but quoting in general is
slightly better) I don't plan on worrying about it.

There are a couple of other minor cleanups, which make no actual
difference (like adding () around the use of the parameter in the
RETURN macro ... which is generally better, but makes no difference
here as the param is always a simple constant.

All the current ATF tests pass.
2019-01-22 14:32:17 +00:00
kre
a672c6e148 NFCI - DEBUG mode only change.
Add tracing of lexical analyser operations.   This is deliberately
kept out of the normal "all on" set as it makes a *lot* of noise
when enabled (especially in verbose mode) - but when needed, it
helps (evidence for which is coming soon).

As usual, no doc, you need the sources (and of course, a specially
built sh to even be able to enable it.)
2019-01-22 13:48:28 +00:00
kre
f2dc4639fa DEBUG mode shell cleanups (NFC for any normal shell).
Add an error DEBUG trace in exraise() (when the shell has detected
some error or signal, and is aborting what it is doing)

Fix an arith error in DEBUG bit assignments (harmless as we haven't
reached the limit of flags yet), and add some missing (recently added)
debug flags so they are turned on when the user (ie: me) asks for
"everything".
2019-01-21 14:29:12 +00:00
kre
9cef82b269 Fix an amazing crazy botch (of mine) when expanding prompt strings
(PS1 etc) which, if the shell were already exiting, and a prompt
were to be expanded (which only really happens if -x is enabled,
and an exit trap is set, so the commands in the trap need PS4
expanded and written, last thing, before the shell exits) the shell
would instead simply exit when it finished expanding PS4 (before
even writing it, or the xtrace output).

There were more conditions required to set up the environment for
this to actually occur (it seems to only happen when the exit trap
is set in a function, called in a command substitution) but that's
unimportant, the code was nonsense.

Problem noticed by Martijn Dekker.

XXX pullup -8
2019-01-21 14:24:44 +00:00
kre
6b3f958390 When we are about to execute something, and the traps are invalid
(which means this is the very first execution in a new subshell)
clear the traps completely, unless the command is "trap".   We were
allowing any special builtin, which was probably harmless, but not
intended.

Also (though not required) permit "command trap" and "eval trap"
and combinations thereof, because they might be useful, and there is
no particular reason why not.   This is all a part of making t=$(trap)
work as POSIX requires, but almost nothing beyond that.  The "trap"
command must be alone (modulo eval and command) in the subshell for
the exception to apply, no t=$(trap; echo) or anything like that.

Martijn Dekker asked for "command trap" to work (no idea why though,
it converts "trap" from being a special builtin, to a normal one,
which means an error won't cause the shell to exit ... if there's
an error, the "trap" command won't do anything useful, and as we
permit no more commands (for this special treatment) the shell is
going to exit anyway, this difference is not really significant.
2019-01-21 14:18:59 +00:00
kre
f499a5853f Add an explanation of the error (warning)
RANDOM initialisation failed
when the shell might print after RANDOM has been reseeded
(which includes at sh startup) the next time RANDOM is accessed.
It indicates that /dev/urandom was not available or did not
provide data - in that case, sh uses a (weak) seed made out of
the pid and time (but otherwise nothing else changes).
2019-01-21 14:09:24 +00:00
kre
dd761e12dc Fix an off by one buffer length problem. Fortunately, it was off by
one in the "safe" way (it was ensuring the buffer always ended in 2 \0
characters ... one is enough.)   This could affect the expansions of
LINENO RANDOM and SECONDS, though only if they have at least 8 digits
(and then, only sometimes).   RANDOM thus is safe, as it never produces
a number with more than 5 digits, you'd need a script with 10000000
lines before there might be an issue with LINENO (and even autoconf
generated scripts don't generally get that bit) and a shell would need
to be running for almost 4 months for SECONDS to climb that high.

Nevertheless: XXX pullup -8.
2019-01-21 13:27:29 +00:00
kre
66ac24c3bf When we exit from running off the end of the input file (which
includes typing ^D) make sure LINENO is set to indicate the last
(actually one past last) line in the input file, rather than
whatever it was set to by the last command that was actually
executed (which could be some line in a function defined in
some other file).

No effect on exit via an explicit exit command - that would already
set the line number correctly.
2019-01-19 14:20:22 +00:00
kre
f04239a697 Allow the decimal radix character '.' to work, regardless of
what the current locale's radix character happens to be,
while still allowing locale specific entry of fractional
seconds (ie: if you're in locale where the radix character
is ',' you san use "sleep 2.5" or "sleep 2,5" and they
accomplish the same thing).

This avoids issues with the "sleep 0.05" in rc.subr which
generated usage messages when a locale that does not use
'.' as its radix character was in use.

Reported on netbsd-users by Dima Veselov, with the problem
diagnosed by Martin Husemann

While here, tighten the arg validity checking (3+4 is
no longer permitted as a synonym of 3) and allow 0.0
to mean the same thing as 0 rather than being an error.

Also, make the SIGINFO reports a little nicer (IMO).

The ATF tests for sleep all pass (not that that means a lot).
2019-01-19 13:27:12 +00:00
kre
cf75e20323 Add some error checking, and stop assuming what the input
will necessarily contain.   Allow defined nodes to use any
intN_t or unintN_t (as well as plain old int) data types
in fields (along with the others that are permitted).

Note: this script is a part of the build procedure for /bin/sh,
the modified version generates the exact same output files
(for the unaltered input specifications) as the previous one
did, hence no visible change is expected (or even possible).

While there is a tiny chance that some host shell will fail
to be able to run this script while building, the script still
uses nothing even slightly exotic, and is much more conservative
than other scripts used during the build process, so there should
be no issues there either.
2019-01-19 13:08:50 +00:00
kre
e8ed7c888d Finish (hopefully) the second half of 1.47 ... make sure
that when traps are marked as invalid, we never use them
for anything except output from the trap command.

Fixes issues where sub-shells of shells which use traps
(eg: to trap SIGPIPE) can end up looping forever if the
signal occurs in a sub-shell (where the trap is supposed
to be reset to its default).   Reported, and mostly
analyzed by Martijn Dekker.
2019-01-18 06:28:09 +00:00
kre
1a9b5ae61d Redo 1.65 in a simpler way. This is the bit rot avoidance code
that is #if 0'd and (still) has never been compiled (most likely
never will be.)

While here, in the same uncompiled code, deal with line number
counting.   Whether this is correct depends upon how this code
is used, and as it never is (and never has been since line numbers
first started being counted), this is somewhat speculative, but
it seems likely to be the correct way to handle things.

NFC (this code is still all #if 0).
2019-01-16 07:14:17 +00:00
kre
20c0839ae2 Don't use quoteflag when deciding if the word after an alias
should be looked up as a potential following alias - if the first
expands to a string that ends with a space (any space, quoted or
not) then the next word is to be treated as an alias candidate.
(POSIX was to specify only unquoted spaces, but is now going to
leave that unspecified, and the "any space" version turns out to
be more useful.

And besides, the quoteflag test didn't work properly, and would
have been very messy to fix ... if in a word (as if we have a
quoted space) it means that the word has been quoted, which meant
that quoted spaces were correctly detected, but it outside a word,
it just means that the previous word was quoted, so it would sometimes
reject alias lookup on the next word in cases where it is unquestioned
it should be done.
2019-01-15 14:23:56 +00:00
kre
be7c7b5cbb pgetc_linecont() needs to use pgetc() rather than pgetc_macro()
so the fake char returned by the latter when an alias ends (which
is there so we can correctly avoid alias recursion) is correctly
ignored where it is not wanted.
2019-01-15 14:17:49 +00:00
kre
39225745fe Correct an (old) typo in a comment. NFC - it is just a comment. 2019-01-09 11:09:16 +00:00
kre
ed405f1d4e Fix the code taken from FreeBSD 2 revisions back, which fixed
aliases, to actually do what it was supposed to do, and not just
come close by accident.   (How broken this was, while still seeming
to work perfectly most of the time was truly amazing!)

This corrects the behaviour of an alias defined with a blank char
as the last of its value, to correctly do an alias lookup on the
word that follows the alias.
2019-01-09 11:08:09 +00:00
kre
6046dde455 Update some dead (#if 0'd) code that is never called to
cope with the changes made in the previous revision, in an
attempt to avoid bit rot.

Untested (uncompiled) - though it should work.

NFC: this change doesn't get compiled, let alone used.
2019-01-09 11:04:54 +00:00
kre
a559cfeaae A similar fix to that added in 1.169 of eval.c, but here for when
processing command substitutions.   If there is an error while processing,
the any pending queued input should be discarded.   From FreeBSD.
2019-01-09 10:59:20 +00:00
kre
a582e232a9 When an error occurs in a builtin from which we do not exit
(a normal builtin, though those are not genrally an issue for
this problem, or a special builtin that has been prefixed by "command")
make sure that we discard any pending input that might have been
queued up, but not yet processed.

We had the mechanism to fix this from when expansion of PS1 etc
was added (which has a similar problem to deal with) - all taken
from FreeBSD - but did not bother to use it here until now...

This fixes an error detected by newly added ATF tests of the eval
builtin, where
	eval 'syntax error
		another command'
would go ahead and evaluate "another command" which should not
happen (note: only when there was a \n between the two).
2019-01-09 10:57:43 +00:00
christos
ab87f51e55 PR/53837: Michael Scholz: src/bin/csh/func.c from current has a superfluous
fprintf
2019-01-06 01:22:50 +00:00
christos
afec7330e3 put back x in xrealloc 2019-01-05 16:56:25 +00:00
christos
1767ce60a2 Welcome to the 21th century csh: retire "ptr_t" now that we have "void *" 2019-01-05 16:54:00 +00:00
maya
bd073e6366 Remove Free, s/xfree/free/.
Standard C says that free should be a no-op for a NULL pointer, so
we don't need an extra function to do this.

While here, add an XXX about a wrong sounding comment
2019-01-05 10:51:06 +00:00
maya
e814216d15 Unifdef compatibility for broken realloc.
No binary change
2019-01-04 19:13:58 +00:00
gutteridge
6326f9f26c Clarify a sentence about the error handling of cp(1)'s -p option.
Addresses PR bin/40336.
2018-12-23 01:29:23 +00:00
kre
af2cfdd7cf Reverse a decision made when the printsignals() routines from
kill and sh were merged so that the shell (for trap -l) and
kill (for kill -l) can use the same routine, and site that function
in the shell, rather than in kill (use the code that is in kill as
the basis for that routine).   This allows access to sh internals,
and in particular to the posix option, so the builtin kill can
operate in posix mode where the standard requires just a single
character (space of newline) between successive signal names (and
we prefer nicely aligned columns instead)..

In a SMALL shell, use the ancient sh printsignals routine instead,
it is smaller (and very much dumber).

/bin/kill still uses the routine that is in its source, and is
not posix compliant.   A task for some other day...
2018-12-12 20:22:43 +00:00
kre
0e129c91e0 More fixes for the SYNPOSIS of the readonly built-in.
The SYNOPSIS for "readonly -q" cannot have the -q be
optional ...   Also harmonise the output appearance with
that of the export command.

wiz: have at it...
2018-12-12 12:56:17 +00:00
kre
80657b2503 Fix Oo Op Oc syntax error (which seemed to work OK to me....)
Pointed out by wiz@
2018-12-12 12:30:59 +00:00
kre
95f39b9c19 The time has come, the Walrus said... (but no shoes,
ships, or sealing wax required).   No oysters either.

I have not taken the Magical Mystery Tour, but I
say it anyway...
2018-12-12 12:16:42 +00:00
kre
150d8297c3 Implement:
readonly -q VAR...
	readonly -p VAR...
	export -q [-x] VAR...
	export -p [-x] VAR...

all available only in !SMALL shells - and while here, limit
"export -x" to full sized shells as well.

Also, do a better job of arg checking and validating of the
export and readonly commands (which is really just one built-in)
and of issuing error messages when something bogus is detected.

Since these commands are special builtin commands, any error
causes shell exit (for non-interactive shells).
2018-12-12 11:51:33 +00:00
kre
64c9a713d8 Fix a botch made in 1.70 (a bit over a week ago) where
var=foo; readonly var=new
now fails.

If var was already set, an attempt to make it readonly, and assign it
a new value at the same time, failed - the readonly flag was set too soon.

Pointed out by Martijn Dekker (thanks).

Also, while here, add a couple of comments.
2018-12-12 07:56:57 +00:00
kre
e3847ee4a9 PR standards/42829
Implement parameter and arithmetic expansion of $ENV
before using it as the name of a file from which to
read startup commands for the shell.   This continues
to happen for all interactive shells, and non-interactive
shells for which the posix option is not set (-o posix).

On any actual error, or if an attempt is made to use
command substitution, then the value of ENV is used
unchanged as the file name.

The expansion complies with POSIX XCU 2.5.3, though that
only requires parameter expansion - arithmetic expansion
is an extension (but for us, it is much easier to do, than
not to do, and it allows some weird stuff, if you're so
inclined....)   Note that there is no ~ expansion (use $HOME).
2018-12-11 13:31:20 +00:00
christos
eb31074ab5 comment out unused. 2018-12-09 17:33:38 +00:00
kre
78cb4ed28f evert previous, linux build problem confirmed fixed by
update to mkinit.sh (to 1.10).

Or more correctly, revert & fix - turns out that there was an off by one
(failure to adjust for other changes -- in a value printed by debug mode
trace output).

NFC.
2018-12-05 22:25:38 +00:00
kre
02e7608416 Use printf, rather than echo (via echo redefined as a function that
uses printf for simplicity).

This script runs using the build host's shell, and echo, and so must
deal with all of the absurdity that different versions of echo dumb
upon us.

This is the underlying cause of the linux build failure that gson@ reported.
2018-12-05 09:20:18 +00:00
kre
24e0ca1d07 NFC (except that it should, I am guessing, fix compilation on
some versions of liux) - DEBUG mode change:  Delete a (relatively new)
trace point (temporarily anyway) which mkinit (a script run using the
host's /bin/sh) apparently cannot handle correctly on (some release of)
linux (it is fine with the NetBSD shell).

I don't know which linux version has a shell with this problem
(or whether it is a mkinit issue that only works by fluke on NetBSD)

Problem reported by gson@
2018-12-05 03:14:28 +00:00
kre
370f00cafc Alter a design botch when magic (self modifying) variables
were added to sh ... in other shells, setting such a variable
(for most of them) causes it to lose its special properties,
and act the same as any other variable.   I had assumed that
was just implementor laziness...   I was wrong.

From now on the NetBSD shell will act like the others, and if vars
like HOSTNAME (and SECONDS, etc) are used as variables in a script
or whatever, they will act just like normal variables (and unless
this happens when they have been made local, or as a variable-assignment
as a prefix to a command, the special properties they would have had
otherwise are lost for the remainder of the life of the (sub-)shell
in which the variables were set).

Importing a value from the environment counts as setting the
value for this purpose (so if HOSTNAME is set in the environment,
the value there will be the value $HOSTNAME expands to).

The two exceptions to this are LINENO and RANDOM.   RANDOM
needs to be able to be set to (re-)set its seed.  LINENO needs to
be able to be set (at least in the "local" command) to achieve
the desired functionality.   It is unlikely that any (sane) script
is going to want to use those two as normal vars however.

While here, fix a minor bug in popping local vars (fn return) that need
to notify the shell of changes in value (like PATH).

Change sh(1) to reflect this alteration.  Also add doc of the
(forgotten) magic var EUSER (which has been there since the others
were added), and add a few more vars (which are documented
in other places in sh(1) - like ENV) into the defined or used
variable list (as well as wherever else they appear).

XXX pullup -8
2018-12-04 14:03:30 +00:00
martin
75d47b0d94 Make pendingsigs forward declaration match the definition. 2018-12-03 10:53:29 +00:00
kre
13fc5c1bcd Cleanup traps a bit - attempt to handle weird uses in traps, such
as traps that issue break/continue/return to cause the loop/function
executing when the trap occurred to break/continue/return, and
generating the correct exit code from the shell including when a
signal is caught, but the trap handler for it exits.

All that from FreeBSD.

Also make
	T=$(trap)
work as it is supposed to (also trap -p).

For now this is handled by the same technique as $(jobs) - rather
than clearing the traps in subshells, just mark them invalid, and
then whenever they're invalid, clear them before executing anything
other than the special blessed "trap" command.   Eventually we will
handle these using non-subshell command substitution instead (not
creating a subshell environ when the commands in a command-sub alter
nothing in the environment).
2018-12-03 06:43:19 +00:00
kre
522d163154 Fix "export -x" (and its consequences) to behave as originally
intended (and as documented) rather than how it has been behaving
(which was not very rational.)   Since it is unlikely that anyone
is using this, the change should be mostly invisible.

While here, a couple of other minor cleanups:
	. One call of geteuid() is enough in choose_ps1()
	. Fix a typo in a comment
	. Improve appearance (whitspace changes) in find_var()
2018-12-03 06:42:25 +00:00
kre
c9f333ad14 Yet another foray into the mysterious world of $@ -- this time
to fix the (unusual) idiom "${1+$@}"  (the quotes are part of it).
This seems to have broken about 5 or 6 years ago (somewhere
between -6 and -7), I believe.

Note this is not the same as "$@" and also not the same as ${1+"$@"}
(much more common idioms) which both worked.

Also attempt to deal with "" more correctly, especially when it
appears adjacent to "$@" (or one of the similar constructs.)

This stuff is still all as ugly and hackish (and fragile) as is
possible to imagine, but in an effort to allow some of the weirdness
to eventually go away, the parser output has been made more
regular and all quoted (parts of) words always now start with
CTLQUOTEMARK and end with CTLQUOTEEND regardless of where the
quotes appear.

This allows us to tell the difference between """$@" and "$@"
which was impossible before - yet they are required to generate
different output when there are no args (when "$@" simply vanishes).

Needless to say that change had ramifications all over the place.
To simplify any similar change in the future, there are some new
macros that can generally be used to detect the "noise" data when
processing words, rather than open coding that every time (which
meant that there would *always* be one which missed getting
updated...)

Several other bugs (of my making, and older ones) are also fixed.

The aim is that (aside from anything that is detecting the cases
that were broken before - which were all unlikely uses of sh
syntax) these changes should have no external visible impact.

Sure...
2018-12-03 06:41:30 +00:00
kre
021ba5091c Revamp aliases - as dumb an idea as they are, if we're going
to have them, they should work as documented, not cause core
dumps, reference after free, incorrect replacements, failing
to implement alias after alias, ...

The big comment that ended:
	  This is a good idea ------- ***NOT***
and the hack it was describing are gone.

Note that most of this was from original CVS version 1.1
code (ie: came from the original import, even before 4.4-Lite
was merged.   That is, May 1994.  And no-one in 24.5 years
noticed (or at least complained about)  all the bugs (or at
least, most of them)).

With these changes, aliases ought to work (if you can call it
that) as they are expected to by POSIX.   Now if only we could
get POSIX to delete them (or make them optional)...

Changes partly inspired by similar changes made by FreeBSD,
(as was the previous change to alias.c, forgot ack in commit
log for that one, apologies) but done a little differently,
and perhaps with a slightly better outcome.
2018-12-03 06:40:26 +00:00
kre
7f63ac72c5 When forking a child shell, arrange for errors/exit to always unwind
to the main handler, rather than wherever the parent shell would go.

nb: not needed for vfork(), after vfork() we never go that path - which
is good or we'd be corrupting the parent's handler.

This allows the child to always exit (when it should) rather than being
caught up doing something else (and while it would eventually exit, the
status would be incorrect in some cases).

One test is:
	sh -c 'trap "(! :) && echo BUG || echo nobug" EXIT'
from Martijn Dekker

Fix from FreeBSD (missed earlier).

XXX - 2b part of the 48875 pullup to -8
2018-12-03 02:38:30 +00:00
kre
154c1ed72b Fix the worst of the bugs in alias processing. This has been in sh
since this code was first imported (May 1994) (ie: before 4.4-Lite)

There is (much) more coming soon (the big ugly comment is going away).

This one has been separated out, as it can easily cause sh
core dumps, so needs:

XXX pullup-8

(the other changes to aliases probably will not get that.)
2018-12-02 10:27:58 +00:00
kre
17e8cd6768 Rename the internal function "makename" to "makeword" to better reflect
what it actually does (makearg would have been an alternative).
While here, notice the one remaining place where it should have been
used, but was left open coded, and consume that one.

NFCI.
2018-12-01 07:02:23 +00:00
kre
ae1a47886f NFC. Need a grain of const 2018-12-01 01:21:06 +00:00
kre
c074191fdd NFC. KNF. return (x) -> return x 2018-12-01 01:20:05 +00:00
kre
d02195817d It is not only the EXIT trap we need to check for when deciding no
fork is required, but any trap   (dumb mistake...)

XXX - include in 48875 pullup to -8
2018-11-30 23:22:45 +00:00
christos
a4c163faee Add base-256 decoding support (Micha Gorny) 2018-11-30 00:53:11 +00:00
kamil
e405eb35a7 Fix typo: O_ALTIO -> O_ALT_IO
Noted by @jbeich via GitHub.
2018-11-26 20:03:39 +00:00
kre
d7fa5d2fb9 Make it be that an empty command is treated as a regular builtin
for the purposes of any redirects it might have -- ie: as posix
requires, make the redirects appear to have been executed in a subshell
environment, so if one fails, aside from a diagnositc msg, all the
running script sees is a command that failed ($? != 0), rather
that having the shell exit which used to happen (the empty command was
being treated as a special builtin).

Continue to treat the empty command as special for the purposes of
var assigns it might contain (those are not executed in a sub-shell
and persist) - an error there (eg: assigning to a readonly var) will
continue to cause the shell (non-interactive shell) to exit.

This makes the NetBSD shell behave like all other (reasonably modern)
shells - fix method (not the implementation, details differ) taken from
FreeBSD who fixed this back in early 2010.    Problem pointed out
in (non-list) mail by Martijn Dekker.
2018-11-26 13:47:39 +00:00
kre
00655d6c45 Fix the <> redirection operator, which has been broken since it was
first implemented in response to PR bin/4966  (PR Feb 1998, fix Feb 1999).

The file named should not be truncated.

No other shell truncates the file (<> was added to FreeBSD sh in Oct 2000,
and did not include O_TRUNC) and POSIX certainly does not suggest that
should happen (just that the file is to be created if it does not exist.)

Bug pointed out in off-list e-mail by Martijn Dekker
2018-11-23 23:41:20 +00:00
kre
c43d5e9f9c Handle eval $'continue\ncommand' (and similar) in a loop correctly ...
"command" should not be executed.  (The issue affects multi-line
eval strings only - ie: commands after the next \n are not skipped).

Bug noted by Martijn Dekker in off-list e-mail.

Fix from FreeBSD:
src/bin/sh/eval.c: Revision 272983 Sun Oct 12 13:12:06 2014 UTC by jilles
2018-11-23 23:37:22 +00:00
kre
30bfb81c65 Avoid long option names that differ only in character case.
Change Xtrace (the name) to xlock instead.  Aside from the different
name, there is no change to functionality.
2018-11-23 20:40:06 +00:00
kre
df073671e8 Rationalise (slightly) the way that expansions are processed
to hide meta-characters in the result when the expansion was
in (double) quotes, and so should not be further processed.

Most of this has been OK for a long while, but \ needs hiding
as well, which complicates things, as \ cannot simply be hidden
in the syntax tables as one of the group of random special characters.

This was fixed earlier for simple variable expansions, but
every variety has its own code path ($var uses different code
than $n which is different than $(...), which is different
again from ~ expansions, and also from what $'...' produces).

This could be fixed by moving them all to a common code path,
but that's harder than it seems.  The form in which the data
is made available differs, so one common routine would need
a whole bunch of different "get the next char or indicate end"
methods - probably via passing in an accessor function.
That's all a lot of churn, and would probably slow the shell.

Instead, just make macros for doing the standard tests, and
use those instead of open coding (differently) each time.
This way some of the code paths don't end up forgetting to
handle '\' (which is different than all the others).

This removes one optimisation ... when no escaping is needed
(like just $var (unquoted) where magic chars (think '*') in
the value are intended to remain magic), the code avoided doing
two tests for each char ("do we need escapes" and "is this char
one that needs escaping") by choosing two different syntax
tables (choice made outside the loop) - one of which never
returns the magic "needs escaping" result, and the other does
when appropriate, and then just avoiding the "do we need escapes"
test for each character processed.   Then when '\' was fixed,
there needed to be another test for it, as it cannot (for other
reasons) be the same as all the others for which "this char
need escaping" is true.   So that added a 2nd test for each char...
Not all the code paths were updated.   Hence the bugs...

nb: this is all rarely seen in the wild, so it is no big
surprised that no-one ever noticed.

Now the "use two different syntax tables" is gone (the two
returned the same for '\' which is why '\' needed special
processing) - and in order to avoid two tests for each
char (plus the \ test) we duplicate the loops, one of which
tests each char to see if it needs an escape, the 2nd just
copies them.   This should be faster in the "no escapes"
code path (though that is not the point) and perhaps also
in the "escapes needed" path (no indirect reference to
the syntax table - though that would probably be in a
register) but makes the code slightly bigger.  For /bin/sh
the text segment (on amd64) has grown by 48 bytes.  But
it still uses the same number of 512 byte pages (and hence
also any bigger page size).  The resulting file size
(/bin/sh) is identical before and after.  So is /rescue/sh
(or /rescue/anything-else).
2018-11-18 17:23:37 +00:00
kre
7ba0d30a60 PR bin/53712
Avoid crash from redirect on null compound command.
2018-11-09 02:11:04 +00:00
kre
375f4ceb14 Allow shells forked to run command substitutions while expanding
prompts to exit when they're done, rather than forcing them to
turn into interactive shells and start reading input ...

Completes a part of the previous changes (just 10+ weeks late...)

Should fix the prompt expansion issue reported by Caóc on
current-users.
2018-11-08 18:37:42 +00:00
kre
33a05de6b0 Switch from using two printsignals() functions, one in trap.c
and one in (the included from bin/kill) kill.c and use just
the one in kill.c (which is amended slightly so it can work
the way that trap.c needs it to work).    This one is chosen as
it was a much nicer implementation, and because while kill is
always built into the shell, kill also exists without the shell.

Leave the old implementation #if 0'd in trap.c (but updated to
match the calling convention of the one in kill.c) - for now.

Delete references of sys_signame[] from sh/trap.c and along with
that several uses of NSIG (unfortunately, there are still more)
and replace them with the newer libc functional interfaces.
2018-10-28 18:26:52 +00:00
kre
185226c2be Use strsignal() rather than direct reference to sys_siglist[]
(apart from being cleaner, it also simplifies the code, as
strsignal() never fails ... it also removes one reference to NSIG).
2018-10-28 18:16:01 +00:00
kre
7800af09a4 Change the (commented out) setting of -DDEBUG to the form that
is most likely to be useful if someone other than me wants to
build a DEBUG shell.   NFC (it is a comment in a Makefile!)
2018-10-28 18:13:47 +00:00
kre
a7fa4685d3 Delete the old style (no longer used) DEBUG mode TRACE compat macro
definitions (just to avoid any temptation to ever use them again).

Update a comment which would make no sense without following the
preceding comment which is being deleted with the macros it describes.

While here, remove another comment that referred to events that have
long past as if they were still to come.   Also a grammatical comment
correction - paragraphs start with capital letters...

NFC (even with DEBUG defined).
2018-10-18 05:28:45 +00:00
kre
bc7bc5b29e Remove a DEBUG mode transition mechanism (for the transition from
the ancient DEBUG TRACE() method, to the newer CTRACE() et. al.)
that turns out never really needed committing - the mechanism, and
the code that obsoleted it, were committed together (May 2017).
[It was useful to me while getting to that state...]

NFC.   Not even with DEBUG shells.
2018-10-18 04:44:27 +00:00
kre
e5830c7775 Dynamically detect the way the shell matches \ in a pattern,
and use whatever works for the sh running this script.  Previously
we were using the (broken, and incorrect) method that worked in
old broken NetBSD sh's (and some others) and not the method that
works with the current (fixed) /bin/sh and other correct shells
(like bash).   (For an exotic reason, in the particular use case,
both methods work with ksh93, but it is also generally correct).

This hasn't really mattered, as the difference is only significant
(only causes actual issues - the build fails) when compiling with DEBUG
enabled, which is something that most sane humans would never do, if they
want to retain that sanity.

The problem was detected by Patrick Welche when looking for an
unrelated problem, which was once considered to be a possible sh
problem, but turned out to be something entirely different.

XXX pullup -8
2018-10-18 04:24:43 +00:00
kre
0db5b60ce4 When (about to) send the -x output for the end of a compound command
(which has redirects, and so is included in -x output) use the -x/+x
setting that existed when the comoound started, so if the state of
xtrace changes during the command we don't end up with just half of
the -x output (either the intro, or the conclusion, depending on
which way the change happened).   [this also happens to avoid a core
dump in the previous code, but that could have been done other ways,
this way actually simplifies things (less code)]
2018-10-09 02:43:41 +00:00
rillig
32c623ff75 When listing aliases, sort them alphabetically. 2018-10-07 23:17:52 +00:00
maxv
a51b00c421 Don't display l_wchan, either there is something in l_wmesg and we display
it, or there's nothing and we print "-".
2018-09-19 15:20:39 +00:00
kre
cf19d2f1a5 Begone unspecified test-a ... and be more obious what it was doing. 2018-09-16 22:31:30 +00:00
rillig
a8fe3451b9 Fix "every" typo in quote from The Mythical Man-Month 2018-09-14 05:59:10 +00:00
kre
feb6abd7ba A change in rev 1.91 interacted badly with the way that showjobs()
worked, preventing $(jobs) (and more usefully $(jobs -p) from
working.   Fix that.

XXX pullup -8
2018-09-13 22:12:35 +00:00
kre
df8e475ae0 Allow SMALL (and TINY) builds of test (for SMALL/TINY builds of sh)
which support only the defined modes of operation of test, to allow
the version of sh on small install media be kept as small as possible.
2018-09-13 22:00:58 +00:00
kre
ec9262987a PR standards/34646
Make test(1) always use the POSIX "number of args" evaluation rules
when they apply.

Only fall back to the old expression evaluation when there are more
than 4 args, or when the args given cannot work as a test expression
using the POSIX rules.  That is when the result is unspecified.

Also fix old bug where a string of whitespace is considered to be a
valid number (at least one digit is needed amongst it somewhere...)

XXX pullup -8
2018-09-12 23:33:31 +00:00
kre
3bdbd8da34 Whitespace cleanup from last update. NFC. 2018-09-11 03:30:40 +00:00