POSIX just says "here docs begin after the next newline [token]".
Nothing about "provided it is inside any `` the redirect operator
appears in... As best I can tell, NetBSD now has the only shell to
handle this "correctly" (which raises the question whether it is
correct - but if not, only erroneous scripts are affected.)
This is required by some (probably broken) autoconfigure related
scripts. (from kre@)
following a suggestion from him, the way the fix to PR bin/50993
was implemented has changed a little. There are three steps involved
in processing a here document, reading it, parsing it, and then
evaluating it before applying it to the correct file descriptor for
the command to use. The third of those is not related to this
problem, and has not changed. The bug was caused by combining the
first two steps into one (and not doing it correctly - which would be
hard that way.) The fix is to split the first two stages into
separate events. The original fix moved the 2nd stage (parsing)
to just immediately before the 3rd stage (evaluation.) Jilles
pointed out some unwanted side effects from doing it that way, and
suggested moving the 2nd stage to immediately after the first.
This commit makes that change. The effect is to revert the changes
to expand.c and parser.h (which are no longer needed) and simplify
slightly the change to parser.c. (from kre@)
variable (with its current value set at 20160401) as discussed on
current-users and tech-userlevel. This also includes the necessary
support to implement it properly (particularly the unexportable
part) and adds options to the export command to support unexportable
variables. Also implement the "posix" option (no single letter
equivalent) which gets its default value from whether or not
POSIXLY_CORRECT is set in the environment when the shell starts
(but can be changed just like any other option using -o and +o on
the command line, or the set builtin command.) While there, fix
all uses of options so it is possible to have options that have a
short (one char) name, and no long name, just as it has been possible
to have options with a long name and no short name, though there
are currently none (with no long name). For now, the only use of
the posix option is to control whether ${ENV} is read at startup
by a non-interactive shell, so changing it with set is not usful
- that might change in the future. (from kre@)
when given without braces (ie: $2 etc). Only the first 9 shell
parameters ($1 .. $9) and the special parameter ($0) can be
referenced this way, $10 is ${1}0 not ${10}. Make it so.
This bug brought to notice by Sven Mascheck's web pages which
discuss (among other things) the history of this (and other ash
based) shells .. see http://www.in-ulm.de/~mascheck/ (from kre@)
user is expected. With the previous commits to parser.c, we no
longer need to handle reading here documents in the (massive)
readtoken1() function. That allows its code to be simplified and
made easier to read and understand (several goto's goto goto heaven.
RIP) (from kre@)
a suggestion from him, the way the fix to PR bin/50993 was implemented
has changed a little. There are three steps involved in processing
a here document, reading it, parsing it, and then evaluating it
before applying it to the correct file descriptor for the command
to use. The third of those is not related to this problem, and
has not changed. The bug was caused by combining the first two
steps into one (and not doing it correctly - which would be hard
that way.) The fix is to split the first two stages into separate
events. The original fix moved the 2nd stage (parsing) to just
immediately before the 3rd stage (evaluation.) Jilles pointed out
some unwanted side effects from doing it that way, and suggested
moving the 2nd stage to immediately after the first. This commit
makes that change. The effect is to revert the changes to expand.c
and parser.h (which are no longer needed) and simplify slightly
the change to parser.c. (from kre@)
documents are processed. Now, when first detected, they are
simply read (the only change made to the text is to join lines
ended with a \ to the subsequent line, otherwise end marker detection
does not work correctly (for here docs with an unquoted endmarker
only of course.) This patch also moves the "internal subroutine"
for looking for the end marker out of readtoken1() (which had to
happen as readtoken1 is no longer reading the here doc when it is
needed) - that uses code mostly taken from FreeBSD's sh (thanks!)
and along the way results in some restrictions on what the end
marker can be being removed. We still do not allow all we should.
(from kre@)
Any redirect (or redirects) before a function definition were
allowed by the parser, but otherwise totally ignored. The standard
syntax does not permit redirects there, now, neither do we. (from kre@)
magic string " \t\n" all over the place, slightly improved
syntax error messages, restructured some of the code for
clarity, don't allow IFS to be imported through the environment,
and remove the (never) conditionally compiled ATTY option.
Apart from one or two syntax error messages, and ignoring IFS
if present in the environment, this is intended to have no
user visible changes. (from kre@)
readtoken1() into a real function of its own (inspired by a
similar change made by FreeBSD - the other internal routines
they moved out are expected to move out here as well soon.)
This change helps avoid gcc 5.3 demanded (not required!) volatile
noise which would slow down parsing. (from kre)
(undoing the effect of that commit on syntax.h when it was
being dynamically generated) from 1996. This means that the shell
parser is now locale independent, so scripts that work anywhere will
work consistently everywhere. Inspired by a similar change in
FreeBSD's sh (from 2010) - the original change in the other direction
came from FreeBSD as well.... Note that this does not in any way
add any kind of support for locales to sh (which is a whole different
problem.) (from kre)
ran; i.e. we want this to work:
$ cat succ1
#!/bin/sh
./succ2 6>out
$ cat succ2
#!/bin/sh
echo succ2 >&6
$ ./succ1
And this to fail:
$ cat fail1
#!/bin/sh
exec 6> out
echo "fail1" >&6
./fail2
exec 6>&-
$ cat fail2
#!obj.amd64/sh
echo "fail2" >&6
$ ./fail1
./fail2: 6: Bad file descriptor
XXX: Do we want a -k (keep flag on exec to make redirections not close-on-exec?
quotes ('_' and '.' do not need quoting) and never quote the '=' in
an assignment (or it would not be one.) From kre, with some refactoring
to be blamed to me.
a terminal. The Open Group notes this historic behavior and correctly
notes that it doesn't make much sense. Note also, that mv(1) has
always respected its '-i' regardless of whether the standard input is
a terminal.
From Timo Buhrmester.
