280389 Commits

Author SHA1 Message Date
mrg
57c87668e3 work around various headers not being fully open without some
special #define being present (ie, define them.)
2020-09-05 13:36:49 +00:00
rillig
09d79112a7 make(1): make Hash_Table independent from -funsigned-char
This only makes a difference for Hash_Table keys outside the ASCII
character set, and these are barely used in practice, if at all.

The effects of this change can only be seen in debug mode, when printing
the full contents of the variable namespaces.  In this output, the order
of the entries might change.

All other use cases stay the same as before.
2020-09-05 13:36:25 +00:00
mrg
5db494bf17 fix various merge botches; we may need to re-port the ThreadLister code. 2020-09-05 13:35:55 +00:00
mrg
257335559d avoid always passing NULL to printf()-like functions 2020-09-05 13:35:24 +00:00
rillig
42e06d6e82 make(1): add expected test result for hashing variable names 2020-09-05 13:34:19 +00:00
rillig
6544b03ea1 make(1): add test for intentional hash collision for variable names
Hash collisions may slow down make in certain special situations.  There
is no point though in maliciously triggering such a situation since
anyone who can inject values into makefiles can easily run shell
commands using the :!cmd! modifier or similar mechanisms.  Crafting
variable names just to slow down make is thus not an attack vector.
2020-09-05 12:59:07 +00:00
mrg
c4c51006fe find header files in /usr/include/gcc-9 and the gcc cpu dir. 2020-09-05 12:19:25 +00:00
mrg
fbc31b01a6 endfor not endif 2020-09-05 11:40:56 +00:00
rillig
4cd5ede6f2 add distrib/sets/fmt-list to format the file lists consistently
This program is much more complicated than sort-list in the same
directory.  It takes care of aligning the fields of the lines so that
lines from the same directory are aligned to each other.  This reduces
horizontal jumps for the category and flags fields.
2020-09-05 11:13:07 +00:00
rillig
21e06026f4 fix alignment in ad.sh3 2020-09-05 11:00:18 +00:00
mrg
ecb0fdec1f disable fetching stuff from libsanitizer for now. it needs libstdc++
to build in mknative using gcc makefiles, and our locale changes break
the "normal" build of this.

make sure MKNATIVE_TARGET is always set.  since mknative-gcc doesn't
care about the version any more (we keep two versions for gcc/gcc.old)
simply set it to "gcc".
2020-09-05 10:58:08 +00:00
mrg
8ecbf5f02b port reachover framework to GCC 9. 2020-09-05 09:50:15 +00:00
mrg
ef2e34874f pull out a few more things for GCC 9. 2020-09-05 09:49:32 +00:00
mrg
6df74746d4 port to GCC 9.3. 2020-09-05 09:16:46 +00:00
mrg
654d12c008 merge GCC 9.3.0. 2020-09-05 09:12:22 +00:00
mrg
181254a7b1 initial import of GCC 9.3.0. changes include:
- live patching support
- shell completion help
- generally better diagnostic output (less verbose/more useful)
- diagnostics and optimisation choices can be emitted in json
- asan memory usage reduction
- many general, and specific to switch, inter-procedure,
  profile and link-time optimisations.  from the release notes:
  "Overall compile time of Firefox 66 and LibreOffice 6.2.3 on
  an 8-core machine was reduced by about 5% compared to GCC 8.3"
- OpenMP 5.0 support
- better spell-guesser
- partial experimental support for c2x and c++2a
- c++17 is no longer experimental
- arm AAPCS GCC 6-8 structure passing bug fixed, may cause
  incompatibility (restored compat with GCC 5 and earlier.)
- openrisc support
2020-09-05 07:52:00 +00:00
maxv
60236c8c49 x86: fix several CPUID flags
- Rename: CPUID_PN      -> CPUID_PSN
           CPUID_CFLUSH  -> CPUID_CLFSH
           CPUID_SBF     -> CPUID_PBE
           CPUID_LZCNT   -> CPUID_ABM
           CPUID_P1GB    -> CPUID_PAGE1GB
           CPUID2_PCLMUL -> CPUID2_PCLMULQDQ
           CPUID2_CID    -> CPUID2_CNXTID
           CPUID2_xTPR   -> CPUID2_XTPR
           CPUID2_AES    -> CPUID2_AESNI
   To match the x86 specification and the other OSes.

 - Remove: CPUID_B10, CPUID_B20, CPUID_IA64. They do not exist.
2020-09-05 07:45:44 +00:00
maxv
cfcb8e39af x86: rename PGEX_X -> PGEX_I
To match the x86 specification and the other OSes.
2020-09-05 07:26:37 +00:00
maxv
4a2e4dc388 nvmm: update copyright headers 2020-09-05 07:22:25 +00:00
rillig
55c34e75ce make(1): remove trailing whitespace in -dg1 debug output 2020-09-05 06:46:12 +00:00
rillig
95fadeb4d8 make(1): move test for -dg1 from opt-debug-g1 to opt-debug-graph1 2020-09-05 06:36:40 +00:00
rillig
edc462c8ea make(1): fix test for the MAKEFILE variable
That test had assumed that it would always be run with CURDIR ==
PARSEDIR, which is not the case for ./build.sh.
2020-09-05 06:25:38 +00:00
rillig
5630da511e make(1): add tests for each debug option 2020-09-05 06:20:50 +00:00
maya
517bc6b8ea fix typo 2020-09-05 04:11:10 +00:00
thorpej
d685f9904b Update a comment. 2020-09-05 03:47:16 +00:00
riastradh
13a2b7334d Revert "ufs: Prevent mkdir from choking on deleted directories."
This change made no sense and should not have been committed.
2020-09-05 02:55:38 +00:00
riastradh
4b73975959 ufs: Prevent mkdir from choking on deleted directories.
Fix some missing uvm_vnp_setsize in screw cases while here.
2020-09-05 02:47:48 +00:00
riastradh
44afc3b3f9 genfs_rename: Fix deadlocks in cross-directory cyclic rename.
Reproducer:

A: for (;;) { mkdir("c", 0600); mkdir("c/d", 0600); mkdir("c/d/e", 0600);
    rmdir("c/d/e"); rmdir("c/d"); }
B: for (;;) { mkdir("c", 0600); mkdir("c/d", 0600); mkdir("c/d/e", 0600);
    rename("c", "c/d/e"); }
C: for (;;) { mkdir("c", 0600); mkdir("c/d", 0600); mkdir("c/d/e", 0600);
    rename("c/d/e", "c"); }

Deadlock:

- A holds c and wants to lock d; and either
- B holds . and d and wants to lock c, or
- C holds . and d and wants to lock c.

The problem with these is that genfs_rename_enter_separate in B or C
tried lock order .->d->c->e (in A/B, fdvp->tdvp->fvp->tvp; in A/C,
tdvp->fdvp->tvp->fvp) which violates the ancestor->descendant order
.->c->d->e.

The resolution is to change B to do fdvp->fvp->tdvp->tvp and C to do
tdvp->tvp->fdvp->fvp.  But there's an edge case: tvp and fvp might be
the same (hard links), and we can't detect that until after we've
looked them both up -- and in some file systems (I'm looking at you,
ufs), there is no mere lookup operation, only lookup-and-lock, so we
can't even hold the lock on one of tvp or fvp when we look up the
other one if there's a chance they might be the same.

Fortunately the cases
(a) tvp = fvp
(b) tvp or fvp is a directory
are mutually exclusive as long as directories cannot be hard-linked.
In case (a) we can just defer locking {tvp, fvp} until the end, because
it can't possibly have {fdvp or fvp, tdvp or tvp} as descendants.  In
case (b) we can just lock them in the order fdvp->fvp->tdvp->tvp or
tdvp->tvp->fdvp->fvp if the first one of {fvp, tvp} is a directory,
because it can't possibly coincide with the second one of {fvp, tvp}.

With this change, we can now prove that the locking order is consistent
with the ancestor->descendant partial ordering.  Where two nodes are
incommensurate under that partial ordering, they are only ever locked
by rename and there is only ever one rename at a time.

Proof:

- For same-directory renames, genfs_rename_enter_common locks the
  directory first and then the children.  The order
  directory->child[i] is consistent with ancestor->descendant and
  child[0]/child[1] are incommensurate.

- For cross-directory renames:

  . While a rename is in progress and the fs-wide rename lock is held,
    directories can be created or removed but not changed, so the
    outcome of gro_genealogy -- which, given fdvp and tdvp, returns
    the node N relating fdvp/N/.../tdvp or null if there is none --
    can only transition from finding N to not finding N, if one of
    the directories is removed while any of the vnodes are unlocked.
    Merely creating directories cannot change the ancestry of tdvp,
    and concurrent renames are not possible.

    Thus, if a gro_genealogy determined the operation to have the
    form fdvp/N/.../tdvp, then it might cease to have that form, but
    only because tdvp was removed which will harmlessly cause the
    rename to fail later on.  Similarly, if gro_genealogy determined
    the operation _not_ to have the form fdvp/N/.../tdvp then it
    can't begin to have that form until after the rename has
    completed.

    The lock order is,

    => for fdvp/.../tdvp:
       1. lock fdvp
       2. lookup(/lock/unlock) fvp (consistent with fdvp->fvp)
       3. lock fvp if a directory (consistent with fdvp->fvp)
       4. lock tdvp (consistent with fdvp->tdvp and possibly fvp->tdvp)
       5. lookup(/lock/unlock) tvp (consistent with tdvp->tvp)
       6. lock fvp if a nondirectory (fvp->t* or fvp->fdvp is impossible)
       7. lock tvp if not fvp (tvp->f* is impossible unless tvp=fvp)

    => for incommensurate fdvp & tdvp, or for tdvp/.../fdvp:
       1. lock tdvp
       2. lookup(/lock/unlock) tvp (consistent with tdvp->tvp)
       3. lock tvp if a directory (consistent with tdvp->tvp)
       4. lock fdvp (either incommensurate with tdvp and/or tvp, or
          consistent with tdvp(->tvp)->fdvp)
       5. lookup(/lock/unlock) fvp (consistent with fdvp->fvp)
       6. lock tvp if a nondirectory (tvp->f* or tvp->tdvp is impossible)
       7. lock fvp if not tvp (fvp->t* is impossible unless fvp=tvp)

Deadlocks found by hannken@; resolution worked out with dholland@.

XXX I think we could improve concurrency somewhat -- with a likely
big win for applications like tar and rsync that create many files
with temporary names and then rename them to the permanent one in the
same directory -- by making vfs_renamelock a reader/writer lock: any
number of same-directory renames, or exactly one cross-directory
rename, at any one time.
2020-09-05 02:47:03 +00:00
riastradh
60f4a93e19 tests/fs/vfs/t_renamerace: Test a screw case hannken@ found. 2020-09-05 02:45:22 +00:00
thorpej
39676bd9eb Add siisata. 2020-09-05 01:28:18 +00:00
thorpej
5172d67b23 Build GENERIC with debug symbols, not just GENERIC.MP. 2020-09-05 01:02:02 +00:00
thorpej
27064dd14d Remove the RAWHIDE kernel; there is not need to keep it around. 2020-09-05 00:58:59 +00:00
rillig
919ee589c7 make(1): rename local functions for parsing conditions
The word "get" implies a cheap operation without side effects.  Parsing
instead has lots of side effects, even if it's only that the parsing
position is updated.
2020-09-04 21:08:44 +00:00
rillig
3e8f8a1a29 make(1): migrate get_mpt_arg to Var_ParsePP
This part is covered well by the unit tests.  When I forgot to decrement
the linePtr, several of them failed reliably.
2020-09-04 20:51:01 +00:00
rillig
09567ebca5 make(1): migrate Var_Parse in CondGetArg to Var_ParsePP 2020-09-04 20:32:34 +00:00
rillig
26222804bc make(1): add more explanation for undefined variable expressions 2020-09-04 20:28:15 +00:00
rillig
a1d05e1e9e make(1): re-enable the archive test
The test had failed in the releng build because it assumed it were run
with .CURDIR == .PARSEDIR.  This assumption is true when the tests are
run directly from usr.bin/make, but not when they are run from
tests/usr.bin/make.
2020-09-04 19:03:38 +00:00
rillig
9cc1256284 make(1): use a stack instead of a list for the nested include path
By using a Stack instead of a Lst, the available API is reduced to the
very few functions that are really needed for a stack.  This prevents
accidental misuse (such as confusing Lst_Append with Lst_Prepend) and
clearly communicates what the expected behavior is.

A stack also needs fewer calls to bmake_malloc than an equally-sized
list, and the memory is contiguous.  For the nested include path, all
this doesn't matter, but the type is so generic that it may be used in
other places as well.
2020-09-04 17:59:36 +00:00
rillig
612b32e506 make(1): unexport For_Iterate
By convention, exported functions have an underscore, and static
functions don't.
2020-09-04 17:35:00 +00:00
maxv
8d45928525 nvmm-x86: improve the CPUID emulation
- Mask DTES64, DS_CPL, CID, SDBG, xTPR, PN.
 - B10, B20 and IA64 do not exist, so just remove them.
2020-09-04 17:09:03 +00:00
maxv
8769d1f4bf nvmm: more __read_mostly 2020-09-04 17:08:01 +00:00
maxv
aea9e528d6 nvmm-x86-vmx: improve the handling of CR0
- Flush the guest TLB when certain CR0 bits change.
 - If the guest updates a static bit in CR0, then reflect the change in
   VMCS_CR0_SHADOW, for the guest to get the illusion that the change was
   applied. The "real" CR0 static bits remain unchanged.
 - In vmx_vcpu_{g,s}et_state(), take VMCS_CR0_SHADOW into account.
 - Slightly modify the CR4 handling code, just for more symmetry with CR0.
2020-09-04 17:07:33 +00:00
maxv
70cdbf1a90 nvmm-x86-svm: check the SVM revision
Only revision 1 exists, but check it, for future-proofness.
2020-09-04 17:06:23 +00:00
rillig
60c2e21286 make(1): add test for the special variable MAKEFILE 2020-09-04 17:05:39 +00:00
maxv
4a8f937fe5 Add a few more CPUID flags. 2020-09-04 17:05:09 +00:00
rillig
d4bba47099 make(1): fix expected file for archive test
This test is currently disabled, therefore it didn't fail immediately.
2020-09-04 17:03:17 +00:00
thorpej
59bd389dae Put the MI cpu_data at the beginning of cpu_info so that it is
cache line aligned.
2020-09-04 15:50:09 +00:00
rillig
acf074f0de clean up file lists
- remove trailing whitespace
- remove empty line
- remove typo "htm\tl", that line is repeated
2020-09-04 13:39:50 +00:00
rillig
8bcc5e16ef add missing RCS Id line to lists/xserver/md.evbmips 2020-09-04 12:08:59 +00:00
rillig
8f99095c48 sort file lists with error checking, use lowercase variable name 2020-09-04 12:02:20 +00:00