Commit Graph

3491 Commits

Author SHA1 Message Date
martin 922823ce68 Pull up following revision(s) (requested by kre in ticket #1169):
bin/sleep/sleep.c: revision 1.25

Allow the decimal radix character '.' to work, regardless of
what the current locale's radix character happens to be,
while still allowing locale specific entry of fractional
seconds (ie: if you're in locale where the radix character
is ',' you san use "sleep 2.5" or "sleep 2,5" and they
accomplish the same thing).

This avoids issues with the "sleep 0.05" in rc.subr which
generated usage messages when a locale that does not use
'.' as its radix character was in use.

Reported on netbsd-users by Dima Veselov, with the problem
diagnosed by Martin Husemann

While here, tighten the arg validity checking (3+4 is
no longer permitted as a synonym of 3) and allow 0.0
to mean the same thing as 0 rather than being an error.

Also, make the SIGINFO reports a little nicer (IMO).

The ATF tests for sleep all pass (not that that means a lot).
2019-03-07 16:56:51 +00:00
martin 958f81bb0d Pull up following revision(s) (requested by kre in ticket #1127):
bin/sh/var.h: revision 1.38 (via patch)
	bin/sh/var.c: revision 1.72
	bin/sh/sh.1: revision 1.211 (via patch)

Alter a design botch when magic (self modifying) variables
were added to sh ... in other shells, setting such a variable
(for most of them) causes it to lose its special properties,
and act the same as any other variable.   I had assumed that
was just implementor laziness...   I was wrong.

From now on the NetBSD shell will act like the others, and if vars
like HOSTNAME (and SECONDS, etc) are used as variables in a script
or whatever, they will act just like normal variables (and unless
this happens when they have been made local, or as a variable-assignment
as a prefix to a command, the special properties they would have had
otherwise are lost for the remainder of the life of the (sub-)shell
in which the variables were set).

Importing a value from the environment counts as setting the
value for this purpose (so if HOSTNAME is set in the environment,
the value there will be the value $HOSTNAME expands to).
The two exceptions to this are LINENO and RANDOM.   RANDOM
needs to be able to be set to (re-)set its seed.  LINENO needs to
be able to be set (at least in the "local" command) to achieve
the desired functionality.   It is unlikely that any (sane) script
is going to want to use those two as normal vars however.

While here, fix a minor bug in popping local vars (fn return) that need
to notify the shell of changes in value (like PATH).
Change sh(1) to reflect this alteration.  Also add doc of the
(forgotten) magic var EUSER (which has been there since the others
were added), and add a few more vars (which are documented
in other places in sh(1) - like ENV) into the defined or used
variable list (as well as wherever else they appear).

XXX pullup -8
2018-12-07 13:23:49 +00:00
martin 5cd696825a Pull up following revision(s) (requested by kre in ticket #1126):
bin/sh/alias.c: revision 1.19

Fix the worst of the bugs in alias processing.   This has been in sh
since this code was first imported (May 1994) (ie: before 4.4-Lite)

There is (much) more coming soon (the big ugly comment is going away).

This one has been separated out, as it can easily cause sh
core dumps, so needs:
XXX pullup-8

(the other changes to aliases probably will not get that.)
2018-12-07 13:14:42 +00:00
martin 33876daeac Pull up following revision(s) (requested by kre in ticket #1125):
bin/sh/redir.c: revision 1.61

Fix the <> redirection operator, which has been broken since it was
first implemented in response to PR bin/4966  (PR Feb 1998, fix Feb 1999).

The file named should not be truncated.

No other shell truncates the file (<> was added to FreeBSD sh in Oct 2000,
and did not include O_TRUNC) and POSIX certainly does not suggest that
should happen (just that the file is to be created if it does not exist.)

Bug pointed out in off-list e-mail by Martijn Dekker
2018-12-07 13:12:02 +00:00
martin f52b6c3485 Pull up following revision(s) (requested by kre in ticket #1086):
bin/sh/parser.c: revision 1.152

PR bin/53712

Avoid crash from redirect on null compound command.
2018-11-12 08:46:54 +00:00
martin 1ca517892c Pull up following revision(s) (requested by kre in ticket #1067):
bin/sh/mkinit.sh: revision 1.9

Dynamically detect the way the shell matches \ in a pattern,
and use whatever works for the sh running this script.  Previously
we were using the (broken, and incorrect) method that worked in
old broken NetBSD sh's (and some others) and not the method that
works with the current (fixed) /bin/sh and other correct shells
(like bash).   (For an exotic reason, in the particular use case,
both methods work with ksh93, but it is also generally correct).

This hasn't really mattered, as the difference is only significant
(only causes actual issues - the build fails) when compiling with DEBUG
enabled, which is something that most sane humans would never do, if they
want to retain that sanity.

The problem was detected by Patrick Welche when looking for an
unrelated problem, which was once considered to be a possible sh
problem, but turned out to be something entirely different.

XXX pullup -8
2018-10-21 12:00:32 +00:00
martin e61ca467d4 Pull up following revision(s) (requested by kre in ticket #1050):
bin/sh/jobs.c: revision 1.101

A change in rev 1.91 interacted badly with the way that showjobs()
worked, preventing $(jobs) (and more usefully $(jobs -p) from
working.   Fix that.

XXX pullup -8
2018-10-09 09:51:57 +00:00
martin a1e25157b9 Pull up following revision(s) via patch (requested by kre in ticket #1015):
bin/sh/expand.c: revision 1.124
	bin/sh/expand.c: revision 1.127
	bin/sh/parser.c: revision 1.148
	bin/sh/parser.c: revision 1.149
	bin/sh/syntax.c: revision 1.6
	bin/sh/syntax.h: revision 1.9 (partial)

First pass at fixing some of the more arcane pattern matching
possibilities that we do not currently handle all that well.

This mostly means (for now) making sure that quoted pattern
magic characters (as well as quoted sh syntax magic chars)
are properly marked, so they remain known as being quoted,
and do not turn into pattern magic.   Also, make sure that an
unquoted \ in a pattern always quotes whatever comes next
(which, unlike in regular expressions, includes inside []
matches),

 -

Part 2 of pattern matching (glob etc) fixes.
Attempt to correctly deal with \ (both when it is a literal,
in appropriate cases, and when it appears as CTLESC when it was
detected as a quoting character during parsing).

In a pattern, in sh, no quoted character can ever be anything other
than a literal character.   This is quite different than regular
expressions, and even different than other uses of glob matching,
where shell quoting is not an issue.

In something like
	ls ?\*.c
the ? is a meta-character, the * is a literal (it was quoted).  This
is nothing new, sh has handled that properly for ever.

But the same happens with
	VAR='?\*.c'
and
	ls $VAR
which has not always been handled correctly.   Of course, in
	ls "$VAR"
nothing in VAR is a meta-character (the entire expansion is quoted)
so even the '\' must match literally (or more accurately, no matching
happens - VAR simply contains an "unusual" filename).  But if it had
been
	ls *"$VAR"
then we would be looking for filenames that end with the literal 5
characters that make up $VAR.

The same kinds of things are requires of matching patterns in case
statements, and sub-strings with the % and # operators in variable
expansions.

While here, the final remnant of the ancient !! pattern matching
hack has been removed (the code that actually implemented it was
long gone, but one small piece remained, not doing any real harm,
but potentially wasting time - if someone gave a pattern which would
once have invoked that hack.)
2018-09-10 15:45:11 +00:00
martin 9aa9c3aaa7 Pull up following revision(s) (requested by sevan in ticket #992):
external/bsd/top/dist/top.1.in: revision 1.11
	sbin/gpt/main.c: revision 1.12
	sbin/amrctl/amrctl.c: revision 1.11
	bin/df/df.c: revision 1.93
	sbin/fsck_ext2fs/fsck_ext2fs.8: revision 1.21
	sbin/fsck_ext2fs/main.c: revision 1.38
	bin/ksh/ksh.Man: revision 1.26
	bin/ln/ln.c: revision 1.40
	bin/df/df.1: revision 1.48
	bin/df/df.1: revision 1.49

Document the WCPU field.

Match SYNOPSIS with usage()

-G cannot be specified alongside -i or -P.
Heads up by <leot>

Add -l to SYNOPSIS

Update usage to include -w

Match sequence as per SYNOPSIS in manual

Remove reference to -c flag which was never implemented.

Remove references to -c flag which was never included.

Add the -T flag to usage()
2018-08-28 13:21:42 +00:00
martin 5ce13ab6e1 Fix merge mishap due to #983 / #989 pullup order. 2018-08-25 17:14:38 +00:00
martin ee4d40d415 Pull up following revision(s) (requested by kre in ticket #989):
bin/sh/eval.c: revision 1.156
	bin/sh/eval.h: revision 1.20
	bin/sh/exec.c: revision 1.53

Fix several bugs in the command / type builtin ( including PR bin/48499 )

1. Make command -pv (and -pV) work (which is not as easy as the PR
   suggests it might be (the "check and cause error" was there because
   it did not work, not in order to prevent it from working).

2. Stop -v and -V being both used (that makes no sense).

3. Stop the "type" builtin inheriting the args (-pvV) that "command" has
   (which it did, as when -v -or -V is used with command, it and type are
    implemented using the same code).

4. make "command -v word" DTRT for sh keywords (was treating them as an error).

5. Require at least one arg for "command -[vV]" or "type" else usage & error.
   Strictly this should also apply to "command" and "command -p" (no -v)
   but that's handled elsewhere, so perhaps some other time.   Perhaps
   "command -v" (and -V) should be limited to 1 command name (where "type"
   can have many) as in the POSIX definitions, but I don't think that matters.

6. With "command -V alias", (or "type alias" which is the same thing),
   (but not "command -v alias") alter the output format, so we get
	ll is an alias for: ls -al
   instead of the old
	ll is an alias for
	ls -al
   (and note there was a space, for some reason, after "for")
   That is, unless the alias value contains any \n characters, in which
   case (something approximating) the old multi-line format is retained.
   Also note: that if code wants to parse/use the value of an alias, it
   should be using the output of "alias name", not command or type.

Note that none of the above affects "command [-p] cmd" (no -v or -V options)
only "command -[vV]" and "type".

Note also that the changes to eval.[ch] are merely to make syspath()
visible in exec.c rather than static in eval.c
2018-08-25 14:48:22 +00:00
martin 0562f970ac Pull up following revision(s) (requested by kre in ticket #988):
bin/sh/parser.c: revision 1.147
	bin/sh/var.c: revision 1.70
	bin/sh/mystring.c: revision 1.18
	bin/sh/options.c: revision 1.53
	bin/sh/histedit.c: revision 1.53

Remove atoi()

Mostly use number() (no longer implemented using atoi()) when an
unsigned integer is required, but use strtoXXX() when a conversion
is wanted, without the possibility or error (like setting OPTIND
and RANDOM).   Always init OPTIND to 1 when sh starts (overriding
anything in environ.)
2018-08-25 14:45:37 +00:00
martin 4271cbf17b Pull up following revision(s) (requested by kre in ticket #987):
bin/sh/trap.c: revision 1.44

PR bin/36532 (perhaps)

This is more or less the same patch as provided in the PR
(just 11 years later, so changed a bit) by woods@...

Since there is no known way to actually cause the reported crash,
we may never know if this change actually fixes anything.   But
even if it doesn't it certainly cannot hurt.

There is a potential race which could possibly explain the issue
(see commentary in the PR) which is not easy to avoid - if that is
the actual cause, this should provide a defence, if not really a fix.
2018-08-25 14:41:21 +00:00
martin 50e9c30575 Pull up following revision(s) (requested by kre in ticket #983):
bin/sh/eval.c: revision 1.158
	bin/sh/eval.h: revision 1.21
	bin/sh/main.c: revision 1.74

PR bin/48875

Revert the changes that were made 19 May 2016 (principally eval.c 1.125)
and the bug fixes in subsequent days (eval.c 1.126 and 1.127) and also
update some newer code that was added more recently which acted in
accordance with those changes (make that code be as it would have been
if the changes now being reverted had never been made).

While the changes made did solve the problem, in a sense, they were
never correct (see the PR for some discussion) and it had always been
intended that they be reverted.   However, in practical sh code, no
issues were reported - until just recently - so nothing was done,
until now...

After this commit, the validate_fn_redirects test case of the sh ATF
test t_redir will fail.   In particular, the subtest of that test
case which is described in the source (of the test) as:

	This one is the real test for PR bin/48875

will fail.

Alternative changes, not to "fix" the problem in the PR, but to
often avoid it will be coming very soon - after which that ATF
test will succeed again.

XXX pullup-8
2018-08-25 14:22:49 +00:00
martin 4caf5e82dd Pull up following revision(s) (requested by kre in ticket #982):
bin/sh/eval.c: revision 1.157

PR bin/42184 PR bin/52687  (detailing the same bug).

Fix "command not found" handling so that the error message
goes to stderr (after any redirections are applied).

More importantly, in

	foo > /tmp/junk

/tmp/junk should be created, before any attempt is made
to execute (the assumed non-existing) "foo".

All this was always true for any command (not found command)
containing a / in its name

	foo/bar >/tmp/junk  2>>/tmp/errs

would have created /tmp/junk, then complained (in /tmp/errs)
about foo/bar not being found.   Now that happens for ordinary
commands as well.

The fix (which I found when I saw differences between our
code and FreeBSD's, where, for the benefit of PR 42184,
this has been fixed, sometime in the past 9 years) is
frighteningly simple.   Simply do not short circuit execution
(or print any error) when the initial lookup fails to
find the command - it will fail anyway when we actually
try running it.   The cost is a (seemingly unnecessary,
except that it really is) fork in this case.

This is what I had been planning, but I expected it would
be much more difficult than it turned out....

XXX pullup-8
2018-08-25 11:45:40 +00:00
martin 99c1c0e99a Pull up following revision(s) (requested by kre in ticket #907):
bin/sh/expand.c: revision 1.122

When matching a char class ([[:name:]]) in a pattern (for filename
expansion, case patterrns, etc) do not force '[' to be a member of
every class.

Before this fix, try:

	case [ in [[:alpha:]]) echo Huh\?;; esac

XXX pullup-8    (Perhaps -7 as well, though that shell version has
much more relevant bugs than this one.)  This bug is not in -6 as
that has no charclass support.
2018-07-13 14:32:01 +00:00
martin 5dfcaccfc4 Pull up following revision(s) (requested by kre in ticket #906):
bin/sh/eval.c: revision 1.155
	bin/sh/mknodes.sh: revision 1.3
	bin/sh/nodes.c.pat: revision 1.14
	bin/sh/exec.h: revision 1.27
	bin/sh/exec.c: revision 1.52

Deal with ref after free found by ASAN when a function redefines
itself, or some other function which is still active.

This was a long known bug (fixed ages ago in the FreeBSD sh) which
hadn't been fixed as in practice, the situation that causes the
problem simply doesn't arise .. ASAN found it in the sh dotcmd
tests which do have this odd "feature" in the way they are written
(but where it never caused a problem, as the tests are so simple
that no mem is ever allocated between when the old version of the
function was deleted, and when it finished executing, so its code
all remained intact, despite having been freed.)

The fix is taken from the FreeBSD sh.

XXX -- pullup-8 (after a while to ensure no other problems arise).
2018-07-13 14:29:15 +00:00
martin 798ca019ef Pull up following revision(s) (requested by christos in ticket #887):
bin/ed/main.c: revision 1.30

Prevent shell execution also in the 'r ! dobad' case, pointed out by
Martijn van Duren, thanks!

XXX: pullup-8
2018-06-22 10:08:22 +00:00
martin bee4462648 Pull up following revision(s) (requested by kre in ticket #804):
bin/sh/parser.c: revision 1.146

PR bin/53201

Don't synerr on
	${var-anything
	more}

The newline in the middle of the var expansion is permitted.

Bug reported by Martijn Dekker from his modernish tests.
XXX pullup-8
2018-05-06 09:32:57 +00:00
martin 4589d5e782 Pull up following revision(s) (requested by kamil in ticket #713):
sys/modules/procfs/Makefile: revision 1.4
	sys/miscfs/procfs/procfs_vfsops.c: revision 1.98
	bin/ps/ps.1: revision 1.108
	sys/compat/linux/arch/i386/linux_ptrace.c: revision 1.32
	sys/miscfs/procfs/procfs_vnops.c: revision 1.198
	sys/kern/sys_ptrace_common.c: revision 1.23
	sys/kern/sys_ptrace_common.c: revision 1.24
	sbin/mount_procfs/mount_procfs.8: revision 1.36
	sys/kern/sys_ptrace_common.c: revision 1.25
	sys/kern/sys_ptrace.c: revision 1.5
	sys/compat/linux/arch/powerpc/linux_ptrace.c: revision 1.30
	sys/sys/proc.h: revision 1.342
	sys/kern/sys_ptrace_common.c: revision 1.26
	sys/miscfs/procfs/procfs_ctl.c: file removal
	sys/kern/sys_ptrace_common.c: revision 1.27
	sys/miscfs/procfs/procfs_subr.c: revision 1.109
	sys/kern/sys_ptrace_common.c: revision 1.28
	sys/secmodel/extensions/secmodel_extensions.c: revision 1.8
	sys/kern/sys_ptrace_common.c: revision 1.29
	sys/sys/ptrace.h: revision 1.62
	sys/compat/netbsd32/netbsd32_signal.c: revision 1.45
	share/man/man9/kauth.9: revision 1.109
	sys/miscfs/procfs/files.procfs: revision 1.12
	sys/compat/netbsd32/netbsd32.h: revision 1.115
	sys/miscfs/procfs/procfs.h: revision 1.72
	sys/compat/netbsd32/netbsd32_ptrace.c: revision 1.5
	sys/kern/kern_sig.c: revision 1.337
	sys/sys/kauth.h: revision 1.75
	sys/sys/sysctl.h: revision 1.224
	sys/kern/sys_ptrace_common.c: revision 1.30
	sys/kern/sys_ptrace_common.c: revision 1.31
	sys/kern/sys_ptrace_common.c: revision 1.32
	sys/kern/sys_ptrace_common.c: revision 1.33
	sys/compat/linux/arch/arm/linux_ptrace.c: revision 1.20
	sys/kern/sys_ptrace_common.c: revision 1.34
	sys/kern/sys_ptrace_common.c: revision 1.36
	sys/kern/kern_proc.c: revision 1.207
	sys/kern/kern_exit.c: revision 1.269
	doc/TODO.ptrace: revision 1.29

Make {s,g}et{db,fp,}regs work again for PK_32 processes
XXX: pullup-8

add disgusting magic to handle compat_netbsd32 as a module.

use process_*reg32 instead of struct *reg32.

Remove the filesystem tracing feature

This is a legacy interface from 4.4BSD, and it was
introduced to overcome shortcomings of ptrace(2) at that time, which are
no longer relevant (performance). Today /proc/#/ctl offers a narrow
subset of ptrace(2) commands and is not applicable for modern
applications use beyond simplistic tracing scenarios.

This removal will simplify kernel internals. Users will still be able to
use all the other /proc files.

This change won't affect other procfs files neither Linux compat
features within mount_procfs(8). /proc/#/ctl isn't available on Linux.

Remove:
 - /proc/#/ctl from mount_procfs(8)
 - P_FSTRACE note from the documentation of ps(1)
 - /proc/#/ctl and filesystem tracing documentation from mount_procfs(8)
 - KAUTH_REQ_PROCESS_PROCFS_CTL documentation from kauth(9)
 - source code file miscfs/procfs/procfs_ctl.c
 - PFSctl and procfs_doctl() from sys/miscfs/procfs/procfs.h
 - KAUTH_REQ_PROCESS_PROCFS_CTL from sys/sys/kauth.h
 - PSL_FSTRACE (0x00010000) from sys/sys/proc.h
 - P_FSTRACE (0x00010000) from sys/sys/sysctl.h

Reduce code complexity after removal of this functionality.

Update TODO.ptrace accordingly: remove two entries about /proc tracing.

Do not keep legacy notes as comments in the headers about removed

PSL_FSTRACE / P_FSTRACE, as this interface had little number of users
(close or equal to zero).
Proposed on tech-kern@.

All filesystem tracing utility users are encouraged to switch to ptrace(2).

Sponsored by <The NetBSD Foundation>

untangle the mess:
- factor out common code
- break each ptrace subcall to its own sub-function
  .. more to come ...
- reduce ifdef ugliness by moving it up top.
- factor out PT_IO and make PT_{READ,WRITE}_{I,D} use it
- factor out PT_DUMPCORE
- factor out sendsig code
  .. more to come ...

handle siginfo requests for ptrace32

ptrace: Partially undo PT_{READ,WRITE}_{I,D} and unbreak these commands

The refactored code did not work and was generating EFAULT.

Sponsored by <The NetBSD Foundation>

Merge the code back; the problem was that since we are reading/writing
to a kernel address for PT_{READ,WRITE}_{I,D} we need the kernel vmspace.
provide separate read and write functions to accomodate register functions
that need a size argument.

don't ignore error from copyout_piod

Use the proper process (the tracee) to get information about lwps and
registers and the tracer for vmspace.

Add new sysctl(3) entry: security.models.extensions.user_set_dbregs

Model this new sysctl(3) entry after "user_set_cpu_affinity" in the same
level of sysctl(3) switches.

Allow to read unconditionally Debug Registers (no change here). This is
convenient as even if a user of a debugger does not use hardware assisted
watchpoints/breakpoints, a debugger can still prompt these values to store
in an internal cache with context of registers. Reading them should have
no security concerns.

Add a paranoid MI switch that prohibits by default setting these registers
by a regular user (non-superuser). Make this switch disabled by default.
There are enough reserved bits out there to allow using them
unconditionally on hardened hosts.

Features shipped with Debug Registers are optional features in debuggers.
There is no reduction in elementary functionality.

Reviewed by <christos>

Sponsored by <The NetBSD Foundation>
2018-04-12 13:42:48 +00:00
snj 3f4bc30989 Pull up following revision(s) (requested by christos in ticket #699):
bin/ed/ed.1: 1.32-1.33
	bin/ed/main.c: 1.29
	usr.bin/patch/pch.c: 1.29
Pass -S to ed(1) so that patches containing ! commands don't run commands.
Real cause of CVE-2018-0492:
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=894667)
--
add -S to disable ! commands.
--
Fix date.
2018-04-08 06:04:08 +00:00
snj 072012476f Pull up following revision(s) (requested by kamil in ticket #515):
bin/ps/ps.c: revision 1.89
Fix an unitialized memory read bug in ps(1)
rawcpu of type int, is declared inside main(){} and it can be passed as
uninitialized to setpinfo().
The setpinfo() function has a switch checking the value of rawcpu:
  if (!rawcpu)
    pi[i].pcpu /= 1.0 - exp(ki[i].p_swtime * log_ccpu);
rawcpu is set to 1 with the command line argument "-C".
   -C           Change the way the CPU percentage is calculated by using a
                "raw" CPU calculation that ignores "resident" time (this
                normally has no effect).
Bug reproducible with an invocation: "ps u". It hides with "ps uC".
Initialize rawcpu by default to 0, before the getopt(3) machinery.
Detected with MSan running on NetBSD/amd64.
Sponsored by <The NetBSD Foundation>
2018-02-03 22:11:36 +00:00
martin 4c75b83d58 Pull up following revision(s) (requested by maya in ticket #505):
bin/ksh/history.c: revision 1.18
Use 0600 as the mode for histfile here too.
pointed out by John D. Baker in PR bin/52480
2018-01-22 12:13:15 +00:00
martin 3cabcbd714 Pull up following revision(s) (requested by kre in ticket #384):
bin/sh/output.c: revision 1.37
Improve quoting in xtrace (-x) output ... if a string ("word") to be
output includes a single quote (') then see if using double-quotes
to quote it is reasonable (if no chars that are magic in " also appear).
If so, and if the string is not entirely the ' character, then
use " quoting.  This avoids some ugly looking results (occasionally).
Also, fix a bug introduced about 20 months ago where null strings
in xtrace output are dropped, instead of made explicit ('').
To observe this, before you get the fix: set -x; echo ''   (or similar.)
Move a comment from the wrong place to the right place.
2017-11-23 13:26:01 +00:00
snj 61ef62d39a Pull up following revision(s) (requested by kre in ticket #355):
bin/sh/parser.c: revision 1.145
PR bin/52715
Correct a (relatively harmless) use after free in prompt expansion
processing [detected by asan.]
Relatively harmless: as (while incorrect) the way the data is (was)
used more or less guaranteed that the buffer contents would be
unaltered until well after they are (were) no longer wanted (this
is the expanded prompt string, it is just output (or copied into
libedit internal storage) and forgotten.
This should make no visible difference to anyone (not using asan or
similar.)
2017-11-17 20:33:53 +00:00
martin 2527d0adc4 Pull up following revision(s) (requested by kre in ticket #337):
bin/sh/jobs.c: revision 1.91 (patch)


PR bin/52640  PR bin/52641

Don't delete jobs from the jobs table merely because they finished,
if they are not the job we are waiting upon.   (bin/52640 part 1)

In a sub-shell environment, don't allow wait to find jobs from the
parent shell that had already exited (before the sub-shell was
created) and return status for them as if they are our children.
(bin/52640 part 2)

Don't have the "jobs" command also be an implicit "wait" command
in non-interactive shells.  (bin/52641)

Use WCONTINUED (when it exists) so we can report on stopped jobs that
"mysteriously" move back to running state without the user issuing
a "bg" command (eg: kill -CONT <pid>)   Previously they would keep
being reported as stopped until they exited.
When a job is detected as having changed status just as we're
issuing a "jobs" command (i.e.: the change occurred between the last
prompt and the jobs command being entered) don't report it twice,
once from the status change, and then again in the jobs command
output.   Once is enough (keep the jobs output, suppress the other).

Apply some sanity to the way jobs_invalid is processed - ignore it
in getjob() instead of just ignoring it most of the time there, and
instead always check it before calling getjob() in situations where
we can handle only children of the current shell.  This allows the
(totally broken) save/clear/restore of jobs_invalid in jobscmd() to
be done away with (previously an error while in the clear state would
have left jobs_invalid incorrectly cleared - shouldn't have mattered
since jobs_invalid => subshell => error causes exit, but better to be safe).

Add/improve the DEBUG more tracing.
2017-11-17 14:56:52 +00:00
snj 15de936bb5 Pull up following revision(s) (requested by kre in ticket #323):
bin/sh/sh.1: revision 1.168
Fix typo: s/one or mode/one or more/
2017-10-25 07:03:10 +00:00
snj 9fead4405d Pull up following revision(s) (requested by kre in ticket #310):
bin/sh/expand.c: revision 1.121
	bin/sh/sh.1: revision 1.167 via patch
Three fixes and a change to ~ expansions
1. A serious bug introduced 3 1/2 months ago (approx) (rev 1.116) which
   broke all but the simple cases of ~ expansions is fixed (amazingly,
   given the magnitude of this problem, no-one noticed!)
2. An ancient bug (probably from when ~ expansion was first addedin 1994, and
   certainly is in NetBSD-6 vintage shells) where ${UnSeT:-~} (and similar)
   does not expand the ~ is fixed (note that ${UnSeT:-~/} does expand,
   this should give a clue to the cause of the problem.
3. A fix/change to make the effects of ~ expansions on ${UnSeT:=whatever}
   identical to those in UnSeT=whatever   In particular, with HOME=/foo
   ${UnSeT:=~:~} now assigns, and expands to, /foo:/foo rather than ~:~
   just as VAR=~:~ assigns /foo:/foo to VAR.   Note this is even after the
   previous fix (ie: appending a '/' would not change the results here.)
   It is hard to call this one a bug fix for certain (though I believe it is)
   as many other shells also produce different results for the ${V:=...}
   expansions than  they do for V=... (though not all the same as we did).
   POSIX is not clear about this, expanding ~ after : in VAR=whatever
   assignments is clear, whether ${U:=whatever} assignments should be
   treated the same way is not stated, one way or the other.
4. Change to make ':' terminate the user name in a ~ expansion in all cases,
   not only in assignments.   This makes sense, as ':' is one character that
   cannot occur in user names, no matter how otherwise weird they become.
   bash (incl in posix mode) ksh93 and bosh all act this way, whereas most
   other shells (and POSIX) do not.   Because this is clearly an extension
   to POSIX, do this one only when not in posix mode (not set -o posix).
2017-10-25 06:51:36 +00:00
martin 37614711f2 Pull up following revision(s) (requested by kre in ticket #250):
bin/sh/var.c: revision 1.67
Fix a bug noticed by Soren Jacobsen running the netbsd-6-0 build.sh which
causes a core dump in some exotic circumstances (when restoring local
variables when a function returns).  ("build.sh makewrapper" exposed it.)
This was introduced in 1.63 - not as part of the substance of that
change (addition) but as an unrelated "must be the right thing to do"
cleanup, which wasn't...
2017-08-31 11:43:44 +00:00
snj 7346e33f26 Pull up following revision(s) (requested by ginsbach in ticket #208):
bin/rmdir/rmdir.c: revision 1.27
PR/48182: Fix rmdir -p handling of top-level (root) directory.
2017-08-14 23:54:30 +00:00
snj b8efdc333f Pull up following revision(s) (requested by kre in ticket #199):
bin/sh/input.c: revision 1.61
	bin/sh/parser.c: revision 1.143
PR bin/52458
Avoid mangling history when editing is enabled, and the prompt contains a \n
Also, allow empty input lines into history when they are being appended to
a previous (partial) command (but not when they would just make an empty entry)
.
For all the gory details, see the PR.
Note nothing here actually makes prompts containing \n work correctly
when editing is enabled, that's a libedit issue, which will be addressed
some other time.
2017-08-09 05:35:18 +00:00
snj b2d141e964 Pull up following revision(s) (requested by kre in ticket #103):
bin/kill/kill.c: 1.28
	bin/sh/Makefile: 1.111-1.113
	bin/sh/arith_token.c: 1.5
	bin/sh/arith_tokens.h: 1.2
	bin/sh/arithmetic.c: 1.3
	bin/sh/arithmetic.h: 1.2
	bin/sh/bltin/bltin.h: 1.15
	bin/sh/cd.c: 1.49-1.50
	bin/sh/error.c: 1.40
	bin/sh/eval.c: 1.142-1.151
	bin/sh/exec.c: 1.49-1.51
	bin/sh/exec.h: 1.26
	bin/sh/expand.c: 1.113-1.119
	bin/sh/expand.h: 1.23
	bin/sh/histedit.c: 1.49-1.52
	bin/sh/input.c: 1.57-1.60
	bin/sh/input.h: 1.19-1.20
	bin/sh/jobs.c: 1.86-1.87
	bin/sh/main.c: 1.71-1.72
	bin/sh/memalloc.c: 1.30
	bin/sh/memalloc.h: 1.17
	bin/sh/mknodenames.sh: 1.4
	bin/sh/mkoptions.sh: 1.3-1.4
	bin/sh/myhistedit.h: 1.12-1.13
	bin/sh/nodetypes: 1.16-1.18
	bin/sh/option.list: 1.3-1.5
	bin/sh/parser.c: 1.133-1.141
	bin/sh/parser.h: 1.22-1.23
	bin/sh/redir.c: 1.58
	bin/sh/redir.h: 1.24
	bin/sh/sh.1: 1.149-1.159
	bin/sh/shell.h: 1.24
	bin/sh/show.c: 1.43-1.47
	bin/sh/show.h: 1.11
	bin/sh/syntax.c: 1.4
	bin/sh/syntax.h: 1.8
	bin/sh/trap.c: 1.41
	bin/sh/var.c: 1.56-1.65
	bin/sh/var.h: 1.29-1.35
An initial attempt at implementing LINENO to meet the specs.
Aside from one problem (not too hard to fix if it was ever needed) this version
does about as well as most other shell implementations when expanding
$((LINENO)) and better for ${LINENO} as it retains the "LINENO hack" for the
latter, and that is very accurate.
Unfortunately that means that ${LINENO} and $((LINENO)) do not always produce
the same value when used on the same line (a defect that other shells do not
share - aside from the FreeBSD sh as it is today, where only the LINENO hack
exists and so (like for us before this commit) $((LINENO)) is always either
0, or at least whatever value was last set, perhaps by
	LINENO=${LINENO}
which does actually work ... for that one line...)
This could be corrected by simply removing the LINENO hack (look for the string
LINENO in parser.c) in which case ${LINENO} and $((LINENO)) would give the
same (not perfectly accurate) values, as do most other shells.
POSIX requires that LINENO be set before each command, and this implementation
does that fairly literally - except that we only bother before the commands
which actually expand words (for, case and simple commands).   Unfortunately
this forgot that expansions also occur in redirects, and the other compound
commands can also have redirects, so if a redirect on one of the other compound
commands wants to use the value of $((LINENO)) as a part of a generated file
name, then it will get an incorrect value.  This is the "one problem" above.
(Because the LINENO hack is still enabled, using ${LINENO} works.)
This could be fixed, but as this version of the LINENO implementation is just
for reference purposes (it will be superseded within minutes by a better one)
I won't bother.  However should anyone else decide that this is a better choice
(it is probably a smaller implementation, in terms of code & data space then
the replacement, but also I would expect, slower, and definitely less accurate)
this defect is something to bear in mind, and fix.
This version retains the *BSD historical practice that line numbers in functions
(all functions) count from 1 from the start of the function, and elsewhere,
start from 1 from where the shell started reading the input file/stream in
question.  In an "eval" expression the line number starts at the line of the
"eval" (and then increases if the input is a multi-line string).
Note: this version is not documented (beyond as much as LINENO was before)
hence this slightly longer than usual commit message.
A better LINENO implementation.   This version deletes (well, #if 0's out)
the LINENO hack, and uses the LINENO var for both ${LINENO} and $((LINENO)).
(Code to invert the LINENO hack when required, like when de-compiling the
execution tree to provide the "jobs" command strings, is still included,
that can be deleted when the LINENO hack is completely removed - look for
refs to VSLINENO throughout the code.  The var funclinno in parser.c can
also be removed, it is used only for the LINENO hack.)
This version produces accurate results: $((LINENO)) was made as accurate
as the LINENO hack made ${LINENO} which is very good.  That's why the
LINENO hack is not yet completely removed, so it can be easily re-enabled.
If you can tell the difference when it is in use, or not in use, then
something has broken (or I managed to miss a case somewhere.)
The way that LINENO works is documented in its own (new) section in the
man page, so nothing more about that, or the new options, etc, here.
This version introduces the possibility of having a "reference" function
associated with a variable, which gets called whenever the value of the
variable is required (that's what implements LINENO).  There is just
one function pointer however, so any particular variable gets at most
one of the set function (as used for PATH, etc) or the reference function.
The VFUNCREF bit in the var flags indicates which func the variable in
question uses (if any - the func ptr, as before, can be NULL).
I would not call the results of this perfect yet, but it is close.
Unbreak (at least) i386 build .... I have no idea why this built for me on
amd64 (problem was missing prototype for snprintf witout <stdio.h>)
While here, add some (DEBUG mode only) tracing that proved useful in
solving another problem.
Set the line number before expanding args, not after.   As the line_number
would have usually been set earlier, this change is mostly an effective
no-op, but it is better this way (just in case) - not observed to have
caused any problems.
Undo some over agressive fixes for a (pre-commit) bug that did not
need these changes to be fixed - and these cause problems in another
absurd use case.   Either of these issues is unlikely to be seen by
anyone who isn't an idiot masochist...
PR bin/52280
removescapes_nl in expari() even when not quoted,
CRTNONL's appear regardless of quoting (unlike CTLESC).
New sentence, new line. Whitespace.
Improve the (new) LINENO section, markup changes (with thanks to wiz@ for
assistace) and some better wording in a few placed.
I am an idiot...  revert the previous unintended commit.
Remove some left over baggage from the LINENO v1 implementation that
didn't get removed with v2, and should have.   This would have had
(I think, without having tested it) one very minor effect on the way
LINENO worked in the v2 implementation, but my guess is it would have
taken a long time before anyone noticed...
Correct spelling in comments of DEBUG only code...
(Perhaps) temporary fix to pkgtools (cwrappers) build (configure).
Expanding  `` containing \ \n sequences looks to have been giving
problems.   I don't think this is the correct fix, but it will do
no worse harm than (perhaps) incorrectly calculating LINENO in this
kind of (rare) circumstance.   I'll look and see if there should be
a better fix later.
s/volatile/const/ -- wonderful how opposites attract like this.
NFC (normal use) - DEBUG only change, when showing empty arg list don't
omit terminating \n.
Free stack memory in a couple of obscure cases where it wasn't
being done (one in probably dead code that is never compiled, the other
in a very rare error case.)   Since it is stack memory it wasn't lost
in any case, just held longer than needed.
Many internal memory management type fixes.
PR bin/52302   (core dump with interactive shell, here doc and error
on same line) is fixed.   (An old bug.)
echo "$( echo x; for a in $( seq 1000 ); do printf '%s\n'; done; echo y )"
consistently prints 1002 lines (x, 1000 empty ones, then y) as it should
(And you don't want to know what it did before, or why.) (Another old one.)
(Recently added) Problems with ~ expansion fixed (mem management related).
Proper fix for the cwrappers configure problem (which includes the quick
fix that was done earlier, but extends upon that to be correct). (This was
another newly added problem.)
And the really devious (and rare) old bug - if STACKSTRNUL() needs to
allocate a new buffer in which to store the \0, calculate the size of
the string space remaining correctly, unlike when SPUTC() grows the
buffer, there is no actual data being stored in the STACKSTRNUL()
case - the string space remaining was calculated as one byte too few.
That would be harmless, unless the next buffer also filled, in which
case it was assumed that it was really full, not one byte less, meaning
one junk char (a nul, or anything) was being copied into the next (even
bigger buffer) corrupting the data.
Consistent use of stalloc() to allocate a new block of (stack) memory,
and grabstackstr() to claim a block of (stack) memory that had already
been occupied but not claimed as in use.  Since grabstackstr is implemented
as just a call to stalloc() this is a no-op change in practice, but makes
it much easier to comprehend what is really happening.  Previous code
sometimes used stalloc() when the use case was really for grabstackstr().
Change grabstackstr() to actually use the arg passed to it, instead of
(not much better than) guessing how much space to claim,
More care when using unstalloc()/ungrabstackstr() to return space, and in
particular when the stack must be returned to its previous state, rather than
just returning no-longer needed space, neither of those work.  They also don't
work properly if there have been (really, even might have been) any stack mem
allocations since the last stalloc()/grabstackstr().   (If we know there
cannot have been then the alloc/release sequence is kind of pointless.)
To work correctly in general we must use setstackmark()/popstackmark() so
do that when needed.  Have those also save/restore the top of stack string
space remaining.
	[Aside: for those reading this, the "stack" mentioned is not
	in any way related to the thing used for maintaining the C
	function call state, ie: the "stack segment" of the program,
	but the shell's internal memory management strategy.]
More comments to better explain what is happening in some cases.
Also cleaned up some hopelessly broken DEBUG mode data that were
recently added (no effect on anyone but the poor semi-human attempting
to make sense of it...).
User visible changes:
Proper counting of line numbers when a here document is delimited
by a multi-line end-delimiter, as in
	cat << 'REALLY
	END'
	here doc line 1
	here doc line 2
	REALLY
	END
(which is an obscure case, but nothing says should not work.)  The \n
in the end-delimiter of the here doc (the last one) was not incrementing
the line number, which from that point on in the script would be 1 too
low (or more, for end-delimiters with more than one \n in them.)
With tilde expansion:
	unset HOME; echo ~
changed to return getpwuid(getuid())->pw_home instead of failing (returning ~)
POSIX says this is unspecified, which makes it difficult for a script to
compensate for being run without HOME set (as in env -i sh script), so
while not able to be used portably, this seems like a useful extension
(and is implemented the same way by some other shells).
Further, with
	HOME=; printf %s ~
we now write nothing (which is required by POSIX - which requires ~ to
expand to the value of $HOME if it is set) previously if $HOME (in this
case) or a user's directory in the passwd file (for ~user) were a null
STRING, We failed the ~ expansion and left behind '~' or '~user'.
Changed the long name for the -L option from lineno_fn_relative
to local_lineno as the latter seemed to be marginally more popular,
and perhaps more importantly, is the same length as the peviously
existing quietprofile option, which means the man page indentation
for the list of options can return to (about) what it was before...
(That is, less indented, which means more data/line, which means less
lines of man page - a good thing!)
Cosmetic changes to variable flags - make their values more suited
to my delicate sensibilities...  (NFC).
Arrange not to barf (ever) if some turkey makes _ readonly.  Do this
by adding a VNOERROR flag that causes errors in var setting to be
ignored (intended use is only for internal shell var setting, like of "_").
(nb: invalid var name errors ignore this flag, but those should never
occur on a var set by the shell itself.)
From FreeBSD: don't simply discard memory if a variable is not set for
any reason (including because it is readonly) if the var's value had
been malloc'd.  Free it instead...
NFC - DEBUG changes, update this to new TRACE method.
KNF - white space and comment formatting.
NFC - DEBUG mode only change - convert this to the new TRACE() format.
NFC - DEBUG mode only change - complete a change made earlier (marking
the line number when included in the trace line tag to show whether it
comes from the parser, or the elsewhere as they tend to be quite different).
Initially only one case was changed, while I pondered whether I liked it
or not.  Now it is all done...   Also when there is a line tag at all,
always include the root/sub-shell indicator character, not only when the
pid is included.
NFC: DEBUG related comment change - catch up with reality.
NFC: DEBUG mode only change.  Fix botched cleanup of one TRACE().
"b" more forgiving when sorting options to allow reasonable (and intended)
flexibility in option.list format.   Changes nothing for current option.list.
Now that excessive use of STACKSTRNUL has served its purpose (well, accidental
purpose) in exposing the bug in its implementation, go back to not using
it when not needed for DEBUG TRACE purposes.   This change should have no
practical effect on either a DEBUG shell (where the STACKSTRNUL() calls
remain) or a non DEBUG shell where they are not needed.
Correct the initial line number used for processing -c arg strings.
(It was inheriting the value from end of profile file processing) - I didn't
notice before as I usually test with empty or no profile files to avoid
complications.   Trivial change which should have very limited impact.
Fix from FreeBSD (applied there in July 2008...)
Don't dump core with input like sh -c 'x=; echo >&$x' - that is where
the word after a >& or <& redirect expands to nothing at all.
Another fix from FreeBSD (this one from April 2009).
When processing a string (as in eval, trap, or sh -c) don't allow
trailing \n's to destroy the exit status of the last command executed.
That is:
	sh -c 'false
	'
	echo $?
should produce 1, not 0.
It is amazing what nonsense appears to work sometimes... (all my nonsense too!)
Two bugs here, one benign because of the way the script is used.
The other hidden by NetBSD's sort being stable, and the data not really
requiring sorting at all...
So as it happens these fixes change nothing, but they are needed anyway.
(The contents of the generated file are only used in DEBUG shells, so
this is really even less important than it seems.)
Another ancient (highly improbable) bug bites the dust.   This one
caused by incorrect macro usage (ie: using the wrong one) which has
been in the sources since version 1.1 (ie: forever).
Like the previous (STACKSTRNUL) bug, the probability of this one
actually occurring has been infinitesimal but the LINENO code increases
that to infinitesimal and a smidgen... (or a few, depending upon usage).
Still, apparently that was enough, Kamil Rytarowski discovered that the
zsh configure script (damn competition!) managed to trigger this problem.
source .editrc after we initialize so that commands persist!
Make arg parsing in kill POSIX compatible with POSIX (XBD 2.12) by
parsing the way getopt(3) would, if only it could handle the (required)
-signumber and -signame options.  This adds two "features" to kill,
-ssigname and -lstatus now work (ie: one word with all of the '-', the
option letter, and its value) and "--" also now works (kill -- -pid1 pid2
will not attempt to send the pid1 signal to pid2, but rather SIGTERM
to the pid1 process group and pid2).  It is still the case that (apart
from --) at most 1 option is permitted (-l, -s, -signame, or -signumber.)
Note that we now have an ambiguity, -sname might mean "-s name" or
send the signal "sname" - if one of those turns out to be valid, that
will be accepted, otherwise the error message will indicate that "sname"
is not a valid signal name, not that "name" is not.   Keeping the "-s"
and signal name as separate words avoids this issue.
Also caution: should someone be weird enough to define a new signal
name (as in the part after SIG) which is almost the same name as an
existing name that starts with 'S' by adding an extra 'S' prepended
(eg: adding a SIGSSYS) then the ambiguity problem becomes much worse.
In that case "kill -ssys" will be resolved in favour of the "-s"
flag being used (the more modern syntax) and would send a SIGSYS, rather
that a SIGSSYS.    So don't do that.
While here, switch to using signalname(3) (bye bye NSIG, et. al.), add
some constipation, and show a little pride in formatting the signal names
for "kill -l" (and in the usage when appropriate -- same routine.)   Respect
COLUMNS (POSIX XBD 8.3) as primary specification of the width (terminal width,
not number of columns to print) for kill -l, a very small value for COLUMNS
will cause kill -l output to list signals one per line, a very large
value will cause them all to be listed on one line.) (eg: "COLUMNS=1 kill -l")
TODO: the signal printing for "trap -l" and that for "kill -l"
should be switched to use a common routine (for the sh builtin versions.)
All changes of relevance here are to bin/kill - the (minor) changes to bin/sh
are only to properly expose the builtin version of getenv(3) so the builtin
version of kill can use it (ie: make its prototype available.)
Properly support EDITRC - use it as (naming) the file when setting
up libedit, and re-do the config whenever EDITRC is set.
Get rid of workarounds for ancient groff html backend.
Simplify macro usage.
Make one example more like a real world possibility (it still isn't, but
is closer) - though the actual content is irrelevant to the point being made.
Add literal prompt support this allows one to do:
CA="$(printf '\1')"
PS1="${CA}$(tput bold)${CA}\$${CA}$(tput sgr0)${CA} "
Now libedit supports embedded mode switch sequence, improve sh
support for them (adds PSlit variable to set the magic character).
NFC: DEBUG only change - provide an externally visible (to the DEBUG sh
internals) interface to one of the internal (private to trace code) functions
Include redirections in trace output from "set -x"
Implement PS1, PS2 and PS4 expansions (variable expansions, arithmetic
expansions, and if enabled by the promptcmds option, command substitutions.)
Implement a bunch of new shell environment variables. many mostly useful
in prompts when expanded at prompt time, but all available for general use.
Many of the new ones are not available in SMALL shells (they work as normal
if assigned, but the shell does not set or use them - and there is no magic
in a SMALL shell (usually for install media.))
Omnibus manual update for prompt expansions and new variables.  Throw in
some random cleanups as a bonus.
Correct a markup typo (why did I not see this before the prev commit??)
Sort options (our default is 0..9AaBbZz).
Fix markup problems and a typo.
Make $- list flags in the same order they appear in sh(1)
Do a better job of detecting the error in pkgsrc/devel/libbson-1.6.3's
configure script, ie: $(( which is intended to be a sub-shell in a
command substitution, but is an arith subst instead, it needs to be
written $( ( to do as intended.   Instead of just blindly carrying on to
find the missing )) somewhere, anywhere, give up as soon as we have seen
an unbalanced ')' that isn't immediately followed by another ')' which
in a valid arith subst it always would be.
While here, there has been a comment in the code for quite a while noting a
difference in the standard between the text descr & grammar when it comes to
the syntax of case statements.   Add more comments to explain why parsing it
as we do is in fact definitely the correct way (ie: the grammar wins arguments
like this...).
DEBUG and white space changes only.   Convert TRACE() calls for DEBUg mode
to the new style.   NFC (when not debugging sh).
Mostly DEBUG and white space changes.   Convert DEEBUG TRACE() calls to
the new format.   Also #if 0 a function definition that is used nowhere.
While here, change the function of pushfile() slightly - it now sets
the buf pointer in the top (new) input descriptor to NULL, instead of
simply leaving it - code that needs a buffer always (before and after)
must malloc() one and assign it after the call.  But code which does not
(which will be reading from a string or similar) now does not have to
explicitly set it to NULL (cleaner interface.)   NFC intended (or observed.)
DEBUG changes: convert DEBUG TRACE() calls to new format.
ALso, cause exec failures to always cause the shell to exit with
status 126 or 127, whatever the cause.  127 is intended for lookup
failures (and is used that way), 126 is used for anything else that
goes wrong (as in several other shells.)  We no longer use 2 (more easily
confused with an exit status of the command exec'd) for shell exec failures.
DEBUG only changes.  Convert the TRACE() calls in the remaining files
that still used it to the new format.   NFC.
Fix a reference after free (and consequent nonsense diagnostic for
attempts to set readonly variables) I added in 1.60 by incompletely
copying the FreeBSD fix for the lost memory issue.
2017-07-23 14:58:13 +00:00
snj 4154318a3c Pull up following revision(s) (requested by kre in ticket #15):
bin/sh/sh.1: revision 1.148
Fix a typo (or rather a remnant of an earlier intent).
2017-06-09 16:53:39 +00:00
snj 317c0b92fd Pull up following revision(s) (requested by kre in ticket #7):
bin/sh/expand.c: revisions 1.111, 1.112
PR bin/52272 - fix an off-by one that broke ~ expansions.
--
Another arithmetic expansion recordregion() fix, this time
calculate the lenght (used to calculate the end) based upon the
correct starting point.
Thanks to John Klos for finding and reporting this one.
2017-06-05 08:15:16 +00:00
snj 7f672caf80 Pull up following revision(s) (requested by kre in ticket #5):
bin/sh/cd.c: revision 1.48
	bin/sh/eval.c: revision 1.141
	bin/sh/exec.c: revision 1.48
	bin/sh/exec.h: revision 1.25
	bin/sh/mail.c: revisions 1.17, 1.18
	bin/sh/sh.1: revision 1.147
Make cd (really) do cd -P, and not just claim that is what it is doing
while doing a half-hearted, broken, partial, version of cd -L instead.
The latter (as the manual says) is not supported, what's more, it is an
abomination, and should never be supported (anywhere.)
Fix the doc so that the pretense that we notice when a path given crosses
a symlink (and turns on printing of the destination directory) is claimed
no more (that used to be true until late Dec 2016, but was changed).  Now
the print happens if -o cdprint is set, or if an entry from CDPATH that is
not "" or "." is used (or if the "cd dest repl" cd cmd variant is used.)
Fix CDPATH processing: avoid the magic '%' processing that is used for
PATH and MAILPATH from corrupting CDPATH.  The % magic (both variants)
remains undocumented.
Also, don't double the '/' if an entry in PATH or CDPATH ends in '/'
(as in CDPATH=":/usr/src/").  A "cd usr.bin" used to do
chdir("/usr/src//usr.bin").  No more.  This is almost invisible,
and relatively harmless, either way....
Also fix a bug where if a plausible destination directory in CDPATH
was located, but the chdir() failed (eg: permission denied) and then
a later "." or "" CDPATH entry succeeded, "print" mode was turned on.
That is:
	cd /tmp; mkdir bin
	mkdir -p P/bin; chmod 0 P/bin
	CDPATH=/tmp/P:
	cd bin
would cd to /tmp/bin (correctly) but print it (incorrectly).
Also when in "cd dest replace" mode, if the result of the replacement
generates '-' as the path named, as in:
	cd $PWD -
then simply change to '-' (or attempt to, with CDPATH search), rather
than having this being equivalent to "cd -")
Because of these changes, the pwd command (and $PWD) essentially
always acts as pwd -P, even when called as pwd -L (which is still
the default.)   That is, even more than it did before.
Also fixed a (kind of minor) mem management error (CDPATH related)
"whosoever shall padvance must stunalloc before repeating" (and the
same for MAILPATH).
--
If we are going to keep the MAILPATH % hack, then at least do something
rational.  Since it isn't documented, what "rational" is is up for
discussion, but what it did before was not it (it was nonsense...).
2017-06-05 08:10:24 +00:00
kre ea89b1308b DEBUG mode only change. Convert old trace style to new, and add some more.
NFC for any non-DEBUG shell.
2017-06-03 21:52:05 +00:00
kre d3573c91dc NFC: Code style only. Rather than being perverse and adding the
negative of a negative number, just add a positive number instead...
(the previous version came about purely as an accident of the way the
relevant piece of code was added and debugged.... that's my story anyway!)
2017-06-03 20:55:53 +00:00
kre 4dd9bbfe51 The correct usage of recordregion() is (begin, end) not (begin, length).
Fixing this fixes a regression introduced earlier today (UTC) where
arithmetic expressions would be split correctly when the arithmetic
started at the beginning of a word:
	echo $(( expression ))
where "begin" is 0, and so (begin, length) is the same as (begin, begin+length)
(aka: (begin,end) - and yes, "end" means 1 after last to consider).
but did not work correctly when the usage was
	echo XXX$( expression ))
(begin !+ 0) and would only split (some part of) the result of the expression.

This regression was also foung by the new t_fsplit:split_arith
test case added earlier to the ATF tests for sh.
2017-06-03 18:37:37 +00:00
kre 375027ae61 When we record an arithmetic expression ($(( ))) as being quoted,
what matters is the quoting state just before we switch into arithmetic
syntax parsing mode, not the state after...

This fixes the regiression introduced earlier today (UTC) where
quoted arithmetic expressions were being subjected to word splitting.
2017-06-03 18:31:35 +00:00
kre dd6b641408 Fixes to shell expand (that is, $ stuff) from FreeBSD (implemented
differently...)

In particular	${01} is now $1 not $0  (for ${0any-digits})

		${4294967297} is most probably now ""
			(unless you have a very large number of params)
		it is no longer an alias for $1  (4294967297 & 0xFFFFFFFF) == 1

		$(( expr $(( more )) stuff )) is no longer the same as
		$(( expr (( more )) stuff )) which was sometimes OK, as in:
			$(( 3 + $(( 2 - 1 )) * 3 ))
		but not always as in:
			$(( 1$((1 + 1))1 ))
		which should be 121, but was an arith syntax error as
			1((1 + 1))1
		is meaningless.

Probably some more.   This also sprinkles a little const, splits a big
func that had 2 (kind of unrelated) purposes into two simpler ones,
and avoids some (semi-dubious) modifications (and restores) in the input
string to insert \0's when they were needed.
2017-06-03 10:31:16 +00:00
abhinav e796b9d453 Fix typo 2017-06-02 17:42:51 +00:00
kre 44443aa011 Add DEBUG tracing to arithmetic $(( )) parsing & evaluation.
NFC for non-DEBUG shells.
2017-05-29 22:54:07 +00:00
kre 374c12e6d0 Now that the shell is protecting its internal fds properly, moving
them whenever the user tries to step on one, we can change our behaviour
back to what the kernel considers to be that of a well behaved shell
(wrt file descriptor usage).  If our user causes problems, we will soon
move into recalcitrant process territory, but that should normally be
rare.  This should reduce kernel overheads a little.
2017-05-29 22:21:00 +00:00
kre 8ffd1099d3 More DEBUG mode changes. As usual, read the source if you care. 2017-05-29 14:03:23 +00:00
kre 9167dc7b19 NFC (normal builds): DEBUG only change - convert parser to newer trace method.
parser tracing is useful when debugging the parser (which admittedly is
fairly often...) but there is a lot of it, and it gets in the way when
looking at something else.   Now we can turn it off when not wanted.
2017-05-29 10:43:27 +00:00
kre d6a4153fe2 Redo mkoptions.sh .. much better this way, now fully automated
option sorting (no longer required option.list to be manually
sorted by long option name) and properly handles conditional
options.   Cleaner output format as well.

This allows option.list to be reordered to group related options
together ... also added more comments to it.
2017-05-28 14:14:22 +00:00
kre f359a311bc Arrange for set -o and $- output to be sorted, rather than more
or less random (and becoming worse as more options are added.)
Since the data is known at compile time, sort at compile time,
rather than at run time.
2017-05-28 00:38:01 +00:00
kre 2d8874d9a7 More standard (and saner) implementation of the ! reserved word.
Unless the shell is compiled with the (compilation time) option
BOGUS_NOT_COMMAND (as in CFLAGS+=-DBOGUS_NOT_COMMAND) which it
will not normally be, the ! command (reserved word) will only
be permitted at the start of a pipeline (which includes the
degenerate pipeline with no '|'s in it of course - ie: a simple cmd)
and not in the middle of a pipeline sequence (no "cmd | ! cmd" nonsense.)
If the latter is really required, then "cmd | { ! cmd; }" works as
a standard equivalent.

In POSIX mode, permit only one !  ("! pipeline" is ok. "! ! pipeline" is not).
Again, if needed (and POSIX conformance is wanted) "! { ! pipeline; }"
works as an alternative - and is safer, some shells treat "! ! cmd" as
being identical to "cmd" (this one did until recently.)
2017-05-27 11:19:57 +00:00
kre 8df083c104 It turns out that most shells do not do variable value/attribute
inheritance when a variable is declared local, but instead leave
the local var unset (if not given a value) in the function.
Only ash derived shells do inheritance it seems.

So, to compensate for that, and get one step closer to making
"local" part of POSIX, so we can really rely upon it, a compromise
has been suggested, where "local x" is implementation defined
when it comes to this issue, and we add "local -I x" to specify
inheritance, and "local -N x" to specify "not" (something...)
(not inherited, or not set, or whatever you prefer to imagine!)
The option names took a lot of hunting to find something reasonable
that no shell (we know of) had already used for some other purpose...
The I was easy, but 'u' 'U' 'X' ... all in use somewhere.

This implements that (well, semi-) agreement.

While here, add "local -x" (which many other shells already have)
which causes the local variable to be made exported.  Not a lot
of gain in that (since "export x" can always be done immediately
after "local x") but it is very cheap to add and allows more other
scripts to work with out shell.

Note that while 'local x="${x}"' always works to specify inheritance
(while making the shell work harder), "local x; unset x" does not
always work to specify the alternative, as some shells have
"re-interpreted" unset of a local variable to mean something that
would best be described as "unlocal" instead - ie: after the unset
you might be back with the variable from the outer scope, rather
than with an unset local variable.

Also add "unset -x" to allow unsetting a variable without removing
any exported status it has.

There are gazillions of other options that are not supported here!
2017-05-27 06:32:12 +00:00
riastradh ef315f7931 Remove MKCRYPTO option.
Originally, MKCRYPTO was introduced because the United States
classified cryptography as a munition and restricted its export.  The
export controls were substantially relaxed fifteen years ago, and are
essentially irrelevant for software with published source code.

In the intervening time, nobody bothered to remove the option after
its motivation -- the US export restriction -- was eliminated.  I'm
not aware of any other operating system that has a similar option; I
expect it is mainly out of apathy for churn that we still have it.
Today, cryptography is an essential part of modern computing -- you
can't use the internet responsibly without cryptography.

The position of the TNF board of directors is that TNF makes no
representation that MKCRYPTO=no satisfies any country's cryptography
regulations.

My personal position is that the availability of cryptography is a
basic human right; that any local laws restricting it to a privileged
few are fundamentally immoral; and that it is wrong for developers to
spend effort crippling cryptography to work around such laws.

As proposed on tech-crypto, tech-security, and tech-userlevel to no
objections:

https://mail-index.netbsd.org/tech-crypto/2017/05/06/msg000719.html
https://mail-index.netbsd.org/tech-security/2017/05/06/msg000928.html
https://mail-index.netbsd.org/tech-userlevel/2017/05/06/msg010547.html

P.S.  Reviewing all the uses of MKCRYPTO in src revealed a lot of
*bad* crypto that was conditional on it, e.g. DES in telnet...  That
should probably be removed too, but on the grounds that it is bad,
not on the grounds that it is (nominally) crypto.
2017-05-21 15:28:36 +00:00