indeed quicker, it nonetheless failed to actually fill all the requested
memory with the specified value much of the time if a non-aligned start
address was used.
- Remove memory barriers from the atomic ops. I don't understand why those
are there. Is it some architectural thing, or for a CPU bug, or just
over-caution maybe? They're not needed for correctness.
- Have unlikely conditional branches go forwards to help the static branch
predictor.
==> Provide a much more complete set of setters and getters for different
value types in the prop_array_util(3) and prop_dictionary_util(3)
functions.
==> Overhaul the prop_data(3), prop_number(3), and prop_string(3) APIs
to be easier to use and less awkwardly named, Deprecate the old
awkward names, and produce link-time warnings when they are referenced.
==> Deprecate mutable prop_data(3) and prop_string(3) objects. The old
APIs that support them still exist, but will now produce link-time
warnings when used.
==> When the new prop_string(3) API is used, strings are internally
de-duplicated as a memory footprint optimization.
==> Provide a rich set of bounds-checked gettter functions in and a
corresponding set of convenience setters in the prop_number(3) API.
==> Add a new prop_bool_value(3) function that is equivalent to
prop_bool_true(3), but aligned with the new "value" routines in
prop_data(3), prop_string(3), and prop_number(3).
This is intended for 68060:
- GCC does not emit __muldi3() for 68020-40, that have 32 * 32 --> 64 mulul
- mulsl (and moveml), used in this code, are not implemented for 68010
In comparison with that from compiler_rt, this version saves:
- 12% of processing time
- 12 bytes of stack
- 50 bytes of code size
Also, slightly faster, memory saving, and smaller than libgcc version.
By examining with evcnt(9), __muldi3() is invoked more than 1000 times per
sec by kernel, which should justify to introduce assembler version of this
function.
Primary goals:
1. Use cryptography primitives designed and vetted by cryptographers.
2. Be honest about entropy estimation.
3. Propagate full entropy as soon as possible.
4. Simplify the APIs.
5. Reduce overhead of rnd_add_data and cprng_strong.
6. Reduce side channels of HWRNG data and human input sources.
7. Improve visibility of operation with sysctl and event counters.
Caveat: rngtest is no longer used generically for RND_TYPE_RNG
rndsources. Hardware RNG devices should have hardware-specific
health tests. For example, checking for two repeated 256-bit outputs
works to detect AMD's 2019 RDRAND bug. Not all hardware RNGs are
necessarily designed to produce exactly uniform output.
ENTROPY POOL
- A Keccak sponge, with test vectors, replaces the old LFSR/SHA-1
kludge as the cryptographic primitive.
- `Entropy depletion' is available for testing purposes with a sysctl
knob kern.entropy.depletion; otherwise it is disabled, and once the
system reaches full entropy it is assumed to stay there as far as
modern cryptography is concerned.
- No `entropy estimation' based on sample values. Such `entropy
estimation' is a contradiction in terms, dishonest to users, and a
potential source of side channels. It is the responsibility of the
driver author to study the entropy of the process that generates
the samples.
- Per-CPU gathering pools avoid contention on a global queue.
- Entropy is occasionally consolidated into global pool -- as soon as
it's ready, if we've never reached full entropy, and with a rate
limit afterward. Operators can force consolidation now by running
sysctl -w kern.entropy.consolidate=1.
- rndsink(9) API has been replaced by an epoch counter which changes
whenever entropy is consolidated into the global pool.
. Usage: Cache entropy_epoch() when you seed. If entropy_epoch()
has changed when you're about to use whatever you seeded, reseed.
. Epoch is never zero, so initialize cache to 0 if you want to reseed
on first use.
. Epoch is -1 iff we have never reached full entropy -- in other
words, the old rnd_initial_entropy is (entropy_epoch() != -1) --
but it is better if you check for changes rather than for -1, so
that if the system estimated its own entropy incorrectly, entropy
consolidation has the opportunity to prevent future compromise.
- Sysctls and event counters provide operator visibility into what's
happening:
. kern.entropy.needed - bits of entropy short of full entropy
. kern.entropy.pending - bits known to be pending in per-CPU pools,
can be consolidated with sysctl -w kern.entropy.consolidate=1
. kern.entropy.epoch - number of times consolidation has happened,
never 0, and -1 iff we have never reached full entropy
CPRNG_STRONG
- A cprng_strong instance is now a collection of per-CPU NIST
Hash_DRBGs. There are only two in the system: user_cprng for
/dev/urandom and sysctl kern.?random, and kern_cprng for kernel
users which may need to operate in interrupt context up to IPL_VM.
(Calling cprng_strong in interrupt context does not strike me as a
particularly good idea, so I added an event counter to see whether
anything actually does.)
- Event counters provide operator visibility into when reseeding
happens.
INTEL RDRAND/RDSEED, VIA C3 RNG (CPU_RNG)
- Unwired for now; will be rewired in a subsequent commit.
kernel and bootloader for 68010.
They requires a special calling convention to udivsi3, and cannot to be
mixed up in normal routines provided by libgcc or compiler_rt. Although,
there's no problem for using them in a controlled situation, i.e., kernel
and standalone programs.
Note that this does not affect at all m68k ports other than sun2, since
codes generated by gcc do not call these routines.
Assembler files are moved from common/lib/libc/arch/m68k/gen to
sys/lib/libkern/arch/m68k in order not to be compiled in libc.
Revert hack introduced to lib/libc/compiler_rt/Makefile.inc rev 1.37:
http://cvsweb.netbsd.org/bsdweb.cgi/src/lib/libc/compiler_rt/Makefile.inc#rev1.37
Proposed on port-sun2@ with no response...
(Again, this does not affect m68k ports other than sun2.)
http://mail-index.netbsd.org/port-sun2/2020/03/10/msg000102.html
- radix_tree_insert_node(): if the insert failed due to ENOMEM, roll back
any updates made to the tree.
- radix_tree_grow(): either succeed or fail, never make partial adjustments
to the tree.
- radix_tree_await_memory(): allocate & free the maximum possible number of
nodes required by any insertion.
This enables us to load modules depended to __ffssi2.
It is difficult to deal with weak symbols consistently in in-kernel
linker. See explanation by pgoyette on tech-kern:
http://mail-index.netbsd.org/tech-kern/2020/03/09/msg026148.html
Also, we do not currently provide ffs(9) as a kernel routine.
Cherry-pick from FreeBSD:
From 7c1bc5ffc2fa68ddc76e5ea8a3a1a6fdfeee57f0 Mon Sep 17 00:00:00 2001
From: andrew <andrew@FreeBSD.org>
Date: Tue, 28 May 2019 09:12:15 +0000
Subject: [PATCH] Teach the kernel KUBSAN runtime about alignment_assumption
This checks the alignment of a given pointer is sufficient for the
requested alignment asked for. This fixes the build with a recent
llvm/clang.
Sponsored by: DARPA, AFRL
common/lib/libc/stdlib/random.c:482:6 can result in signed integer overflow.
This bug was reported by UBSan runs.
The change has been tested using the following program to generate random numbers
in both the old and the new library and can be used to verify the correctness of the
library after the change.
#include <stdio.h>
#include <stdlib.h>
#define COUNT 1000 * 1000
int
main(void)
{
int i;
FILE *fp = fopen("numbers.txt", "w");
srandom(0xdeadbeef);
for(i = 0; i < COUNT; i++) {
fprintf(fp, "%ld\n", random());
}
fclose(fp);
return 0;
}
Reviewed by: riastradh@ , kamil@
this increases the cache groups from 15 to 63 items in order
to reduce traffic between pool cache layers
this is the same as for other highly frequented pool caches as the pvpool and anonpool
uniformly use 'j' in the user-provided formatting strings instead
of depending on _LP64 to use 'll' or 'l' (and the PRI macros). The
alternative is to parse the printf format manually to determine
which modifier to apply which would make this transparent to the
user (they could still always use '%u' or '%x'), but that's too
painful.
need a non-zero check, substitute with a deterministic bitwise OR of all
values in the node. The structure then becomes cache line aligned.
For each node we now need only touch 2 cache lines instead of 3, which makes
all the operations faster (measured), amortises the cost of not having a
counter, and will avoid intra-pool-page false sharing on MP.
Other architectures do not define them, and so we don't provide a
function declaration in any header.
This means a package may detect it with a link-test and then fail
due to the missing declaration, like sysutils/collectd currently does.
Done this way as aarch64 has not had a release yet. Discussed with releng.
- code that will be unreachable on platforms with
sizeof(double) != sizeof(unsigned long) triggered an valid out
of bounds warning. avoid the error by using sizeof ul.
- also assert that the sizes are the same if entering here.
both from kamil@.
Define _REENTRANT for MKSANITIZER build. This is needed for at least stdio
code. This caused new build issued with duplicated symbols in few places
and rump kernel code picking different code paths borrowed from libc.
Handle all this in one go.
Add bsd.sanitizer.mk to share common code used by programs and libraries.
Switch from realall to beforeinstall target in .syms files. This is more
reliable in MKSANITIZER.
the Matt Thomas rbtree:
- Include rb.c in libnbcompat, and provide a nbcompat sys/rbtree.h
header.
- Make sure libprop's source file include prop_object_impl.h before
anything else, and pull in nbtool_config.h from there.
Tested by simulating such a host system by renaming the host's
<sys/rbtree.h> out of the way (which reproduced the build failure)
and verifying that the host-tool installboot contained the rb_*
functions in its own .text segment.
The F spec is one of those, it should be terminated with \0 just like all
the others (irrelevant that it has no extra data to delimit).
Fix <sys/mman.h> to define the snprintb() format string correctly (include
the missing \0's). Fix the copy of that definition included into
snprintb(3) to match the updated mman.h version (ride the date bump
from the day before yesterday .. this is the same change, just corrected).
Undo the previous snprintb.c change ("off by one" fix) which was an
attempt to make the broken mman.h usage work (and did, but not the way
it should be done). Also, after using the new * format (instead of only
when something has already matched) skip the associated data so we don't
attempt to interpret it as more field specifiers. This func needs lots of TLC!
Fix the ATF tests for snprintb() to not assume that F format is really
exactly like f format, and has data after the field specifier. It doesn't.
Add several more tests (including testing the '*' field operator
recently added).
Compile test only, but seems trivial enough for me.
Fix build error due to test/lib/libc/atomic/t_atomic_cas.
Note that mips32 does not use atomic_cas.S.
Note that _atomic_cas_64 is already exported as atomic_cas_{ulong,prt}_ni.
Fix build error of test/lib/atomic/t_atomic_cas, which is successfully
passed on RPI3B+ now.
changing shared macro doesn't effect other atomic_ops because
(*ptr [+|^] delta) and (delta [+|^] *ptr) have same result.
atomic_sub_*() haven't used because non standard API?
As noted, style has no impact on the comparison of a similar code.
This version is a reimplementation from scratch with no code and no
algirithm (whenever possible) reuse.
Public symbols and struct layout must be kept in sync with the code
generation part. It casues violation of the style like with long filenames.
My previous comment was 'too perfect' and could trigger some unnecessary
attention.
This is a reimplementation of the Undefined Behavior Sanitizer with the
following properties:
- pure and clean-room C implementation,
- no -fsanitize=vpts support, as it requires RTTI support and C++
low-level routies to validate whether C++ objects are compatible
- designed to be used inside libc and known as uUBSan or user-UBSan
- designed to be shared with kernel and known as kUBSan or kernel-UBSan
- designed to be usable with ATF tests as a standalone runtime,
reachable without any MK* switches
- designed to be safer for hardening as it does not have side effects on
executables like writing to a selected location on demand
- controllable with environment variable LIBC_UBSAN with options:
* a - abort on report
* A - do not abort on a report (unless a failure is unrecoverable)
* e - output report to stderr
* E - do not output report on stderr
* l - output report on syslog (LOG_DEBUG | LOG_USER)
* L - do not output report on syslog
* o - output report on stdout
* O - do not output report on stdout
The default options are: "AeLO".
- compatible with Clang (3.8, 7.x) and GCC (6.x) code generation
- all handlers (except =vptr) from Clang/LLVM up to 7svn are supported
Tested with Clang amd64+i386 and GCC amd64+i386.
Do not change the signedness bit with a left shift operation.
Switch to unsigned integer to prevent this.
snprintb.c:178:17, left shift of 1 by 31 places cannot be represented in type 'int'
Detected with micro-UBSan in the user mode.
Do not change the signedness bit with a left shift operation.
Cast to unsigned integer to prevent this.
inet_addr.c:218:20, left shift of 131 by 24 places cannot be represented in type 'int'
Detected with micro-UBSan in the user mode.
Do not change the signedness bit with a left shift operation.
Switch to unsigned integer to prevent this.
cpuset.c:112:18, left shift of 1 by 31 places cannot be represented in type 'int'
Detected with micro-UBSan in the user mode.
- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. (probably more to come,
the set between x86 and sparc us disjoint.)
- port libsanitizer's GetPcSpBp() to sparc, sparc64 and amd64.
- -Wstrict-prototypes is not available for C++, so don't try to
ignore it for C++.
- remove many _DIAGASSERT() checks against not NULL for functions
with arguments with nonnull attributes. in two cases, leave
code behind that should set defaults to "(null)".
- use -Wno-error=frame-address for i386 mcount, as it seems valid
to assume the caller will have a frame.fair
The NetBSD Standard C Library uses internally some of its functions with
a mangled symbol name, usually "_symbol". The internal functions shall not
use the global (public) symbols.
This change eliminates usage of the global changes of the following symbols:
- strlcat -> _strlcat
- sysconf -> __sysconf
- closedir -> _closedir
- fparseln -> _fparseln
- kill -> _kill
- mkstemp -> _mkstemp
- reallocarr -> _reallocarr
- strcasecmp -> _strcasecmp
- strncasecmp -> _strncasecmp
- strptime -> _strptime
- strtok_r -> _strtok_r
- sysctl -> _sysctl
- dlopen -> __dlopen
- dlclose -> __dlclose
- dlsym -> __dlsym
Sponsored by <The NetBSD Foundation>
The rev 1.4 changelog:
> Disable a sanity check output buffer != NULL in _STANDALONE case.
> Some kernels are loaded at address 0x0 by bootloaders and
> output buffer address could be zero in such case.
>
> Fixes "read text" errors on loading install floppy of NetBSD/news68k 4.0,
> reported by KIYOHARA Takashi on port-news68k.
This problem may also affect other m68k ports which use a gzipped install
kernel and kernel text address located at PA 0x0.
Should be pulled up to netbsd-8.
Also, use Xr to refer problib(3) in the HISTORY section instead of Nm.
While parsing the man pages, any .Nm occurrence gets replaced by the
value specified in the NAME section. Referencing individual function
names with .Nm was causing makemandb(8) to replicate the complete
NAME section for every such occurrence. This was leading to an
ugly looking snippet in apropos(1)'s output when searched for
any of these man pages.
simonb