Evaluation of commands goes completely haywire if a file containing
a break/continue/return command outside its "intended" scope is sourced
using a dot command inside its "intended" scope. The main symptom is
not exiting from the sourced file when supposed to, leading to evaluation
of commands that were not supposed to be evaluated. A secondary symptom
is that these extra commands are not evaluated correctly, as some of them
are skipped. Some examples are listed in the How-To-Repeat section.
According to the POSIX standard, this is how it should work:
dot:
The shell shall execute commands from the file in the current
environment.
break:
The break utility shall exit from the smallest enclosing for, while,
or until loop, [...]
continue:
The continue utility shall return to the top of the smallest
enclosing for, while, or until loop, [...]
return:
The return utility shall cause the shell to stop executing
the current function or dot script. If the shell is not currently
executing a function or dot script, the results are unspecified.
It is clear that return should return from a sourced file, which
it does not do. Whether break and continue should work from the sourced
file might be debatable. Because the dot command says "in the current
environment", I'd say yes. In any case, it should not fail in weird
ways like it does now!
The problems occur with return (a) and break/continue (b) because:
1) dotcmd() does not record the function nesting level prior to
sourcing the file nor does it touch the loopnest variable,
leading to either
2 a) returncmd() being unable to detect that it should not set
evalskip to SKIPFUNC but SKIPFILE, or
b) breakcmd() setting evalskip to SKIPCONT or SKIPBREAK,
leading to
3) cmdloop() not detecting that it should skip the rest of
the file, due to only checking for SKIPFILE.
The result is that cmdloop() keeps executing lines from the file
whilst evalskip is set, which is the main symptom. Because
evalskip is checked in multiple places in eval.c, the secondary
symptom appears.
>How-To-Repeat:
Run the following script:
printf "break\necho break1; echo break2" >break
printf "continue\necho continue1; echo continue2" >continue
printf "return\necho return1; echo return2" >return
while true; do . ./break; done
for i in 1 2; do . ./continue; done
func() {
. ./return
}
func
No output should be produced, but instead this is the result:
break1
continue1
continue1
return1
The main symptom is evident from the unexpected output and the secondary
one from the fact that there are no lines with '2' in them.
>Fix:
Here is patch to src/bin/sh to fix the above problems. It keeps
track of the function nesting level at the beginning of a dot command
to enable the return command to work properly.
I also changed the undefined-by-standard functionality of the return
command when it's not in a dot command or function from (indirectly)
exiting the shell to being silently ignored. This was done because
the previous way has at least one bug: the shell exits without asking
for confirmation when there are stopped jobs.
Because I read the standard to mean that break and continue should have
an effect outside the sourced file, that's how I implemented it. For what
it's worth, this also seems to be what bash does. Also laziness, because
this way required no changes to loopnesting tracking. If this is not
wanted, it might make sense to move the nesting tracking to the inputfile
stack.
The patch also does some clean-up to reduce the amount of global
variables by moving the dotcmd() and the find_dot_file() functions from
main.c to eval.c and making in_function() a proper function.
- ansification
- format of output of jobs command (etc)
- job identiers %+, %- etc
- $? and $(...)
- correct quoting of output of set, export -p and readonly -p
- differentiation between nornal and 'posix special' builtins
- correct behaviour (posix) for errors on builtins and special builtins
- builtin printf and kill
- set -o debug (if compiled with DEBUG)
- cd src obj (as ksh - too useful to do without)
- unset -e name, remove non-readonly variable from export list.
(so I could unset -e PS1 before running the test shell...)
the non-vfork case. Having said that, it would be nice if pipelines of
simple commands were vforked too. Right now they are not.
Explain that setpgid() might fail because we are doing it both in the
parent and the child case, because we don't know which one will come
first.
Suspending a pipeline prints %1 Suspended n times where n is the number
of processes, but that was there before. It is easy to fix, but I'll
leave the code alone for now.
Propagate isroot, throughout the eval process and maintain it properly.
Fixes sleep 10 | cat^C not exiting because sleep and cat ended up in
their own process groups, because wasroot was always true in the children.
Plus my changes:
- walking process group fix in foregrounding a job.
- reset of process group in parent shell if interrupted before the wait.
- move INTON lower in the dowait so that the job structure is
consistent.
- error check all setpgid(), tcsetpgrp() calls.
- eliminate unneeded strpgid() call.
- check that we don't belong in the process group before we try to
set it.
Fix from FreeBSD:
growstackblock() sometimes relocates a stack_block considered empty
without properly relocating stack marks referencing that block.
The first call to popstackmark() with the unrelocated stack mark
as argument then causes sh to abort.
Relocating the relevant stack marks seems to solve this problem.
The patch changes the semantics of popstackmark() somewhat. It can
only be called once after a call to setstackmark(), thus cmdloop() in
main.c needs an extra call to setstackmark().
even if the specified file is not in the PATH.
This change enforces security and makes it conform to POSIX.
Closes PR #6794.
I thought Christos committed this but not appeared yet. :)
The setstackmark()/popstackmark() pair in dotcmd(), used for freeing
stack storage possibly allocated by find_dot_file(), is redundant for now
since dotcmd() is surrounded by another pair in evalcommand().
This redundancy, however, may help future modifications
(suggested by Christos).
command not found. Add extra exception type and generalize
error handling routines to take that exception type. Use
a global variable exerrno to keep the last exec error.
When not in a function, it skips the rest of the current input file.
Instances of `return' outside function definitions were previously ignored.
What does joe posix have to say about this?
[fixes PR/1444]
$ cat foo
echo $FOO
$ FOO=1 . ./foo
$ echo $FOO
Did not echo 1 at all. Now variable assignments before sourced scripts
are honored during and after the execution of the script.
christos