to detect race conditions at runtime. It is a variation of TSan that is
easy to implement and more suited to kernel internals, albeit theoretically
less precise than TSan's happens-before.
We do basically two things:
- On every KCSAN_NACCESSES (=2000) memory accesses, we create a cell
describing the access, and delay the calling CPU (10ms).
- On all memory accesses, we verify if the memory we're reading/writing
is referenced in a cell already.
The combination of the two means that, if for example cpu0 does a read that
is selected and cpu1 does a write at the same address, kCSan will fire,
because cpu1's write collides with cpu0's read cell.
The coverage of the instrumentation is the same as that of kASan. Also, the
code is organized in a way similar to kASan, so it is easy to add support
for more architectures than amd64. kCSan is compatible with KCOV.
Reviewed by Kamil.
bsd.own.mk also adds it for gcc only (so we end up with 2 -marm for gcc).
Just add -marm unconditionally in bsd.own.mk in ARM_APCS_FLAGS and remove
it from everywhere else.
2. gcc-8 now warns about thumb internetwork in APCS mode. This does not make
sense because the documentation says that this flag does not make a
difference in apcs mode, but might produce larger code in non-APCS mode.
So we just add it to silence the warnings.
GCC_NO_FORMAT_TRUNCATION -Wno-format-truncation (GCC 7/8)
GCC_NO_STRINGOP_TRUNCATION -Wno-stringop-truncation (GCC 8)
GCC_NO_STRINGOP_OVERFLOW -Wno-stringop-overflow (GCC 8)
GCC_NO_CAST_FUNCTION_TYPE -Wno-cast-function-type (GCC 8)
use these to turn off warnings for most GCC-8 complaints. many
of these are false positives, most of the real bugs are already
commited, or are yet to come.
we plan to introduce versions of (some?) of these that use the
"-Wno-error=" form, which still displays the warnings but does
not make it an error, and all of the above will be re-considered
as either being "fix me" (warning still displayed) or "warning
is wrong."
Define _REENTRANT for MKSANITIZER build. This is needed for at least stdio
code. This caused new build issued with duplicated symbols in few places
and rump kernel code picking different code paths borrowed from libc.
Handle all this in one go.
Add bsd.sanitizer.mk to share common code used by programs and libraries.
Switch from realall to beforeinstall target in .syms files. This is more
reliable in MKSANITIZER.
Add per-program rules to disable ASLR for ASan, TSan and MSan in all
sanitized programs. This flag is not needed for other supported sanitizers.
Without this change, sanitized init(8) dies early on startup.
Approach originally suggested by <joerg>
This adds a more flexible version of SANITIZER_RENAME_SYMBOL.
New symbol allows to specify MKSANITIZER specific renames on per-file and
per-symbol basis.
Also add the other gm20x nouveau firmwares to the source tree
to make it easier to add them for someone who can test them.
Installed if MKNOUVEAUFIRMWARE is set to 'yes'.
This defaults to no except on amd64 and i386 (like for radeon).
MKX11 is used before it was defaulted, leading to eg:
... Malformed conditional (${MKX11} != "no" && ${HAVE_MESA_VER} == "18")
gain knowledge that it defaults to "no" immediately below.
could split this list, but seems uglier.
- change the defaults for MKLLVMRT and MKX11 so we don't have to
force-override later. this makes "build.sh -V MKLLVMRT=no"
work to disable building the LLVM Mesa components.
- in gallium, don't BUILD_RADEON if MKLLVMRT is no.
this stops GL from working, but basic X seems OK enough for Xvideo
to still work on modern radeon (ie, video mostly works fine, though
eg, "mpv -vo gpu" won't. use "mpv -vo xv".)
Stop instrumenting KCOV files with kASan and the other way around.
This fixes booting of the setup of using them together.
In theory the checks could be more fine grained, however there is no good
reason (except extra DIAGNISTIC) for running a kernel sanitizer without
kernel coverage.
Patch by <R3x>
for x86 and evbarm:
- install headers and libEGL
- install xorg-server glamoregl component
- link xorg drivers against gbm/egl
bonus fix:
- fix some wrongly marked compatx11file files
build tested on shark, sgimips, evbarm64-el, amd64 and i386.
run tested on radeon, intel and nvidia on amd64, including
'modesetting' driver on amd64. however, my systems disable
it due to llvmpipe so i'm not sure what is happening.
note vax atf fails same as GCC 6. hppa mostly works but has an
atf issue also unrelated to the compiler version
note ia64 mostly builds, but the kernel fails earlier than GCC 6.
- only build clang or clang-tblgen if MKLLVMCMDS != "no"
- move HAVE_MESA_VER earlier so we can detect mesa 18 and enable
MKLLVM on platforms that need it
- move the clang commands and headers into llvmcmds group
- only build the clang specific LLVM libraries if MKLLVMCMDS != "no"
much of this work came from maya@'s github commit
f90685c11d4460d3098fa35f48b58d1893e974e0, reworked for HAVE_MESA_VER
and llvmcmds vs MKCLANG.
build tested on several platforms and on amd64 and i386 with several
different build options (clang build, gcc build, gcc build with llvm
libs).
- add ${X11SRCDIR.MesaLib.old} and introduce ${X11SRCDIR.Mesa} as an
alias for either the former or the non old. this allows many of
the makefiles to simply use ${X11SRCDIR.Mesa} (but does not really
enable much sharing of makefiles, but reduces their diffs.)
- use mesa-which.mk to define ${OLD_PREFIX} to either "" or ".old",
and to know if to build 'dri7' (.old only.) ${OLD_PREFIX} is used
by other code (eg, LIBDPLIBS) to pick the right subdir.
The KCOV driver implements collection of code coverage inside the kernel.
It can be enabled on a per process basis from userland, allowing the kernel
program counter to be collected during syscalls triggered by the same
process.
The device is oriented towards kernel fuzzers, in particular syzkaller.
Currently the only supported coverage type is -fsanitize-coverage=trace-pc.
The KCOV driver was initially developed in Linux. A driver based on the
same concept was then implemented in FreeBSD and OpenBSD.
Documentation is borrowed from OpenBSD and ATF tests from FreeBSD.
This patch has been prepared by Siddharth Muralee, improved by <maxv>
and polished by myself before importing into the mainline tree.
All ATF tests pass.
platforms where PIC is available, so that linking shared objects with
-pg works, without needing to create a shared library version of the
profiled libraries.
The bsd.sys.mk tools are un-prefixed, since the "right" version is installed
for the platform, as opposed to the tools ones which should be able to
co-exist.
the comment why it is needed was introduced in 2001. We don't want
to duplicate -c which is set in ${COPY} because someone might want
to override that.
It works by tainting memory sources with marker values, letting the data
travel through the kernel, and scanning the kernel<->user frontier for
these marker values. Combined with compiler instrumentation and rotation
of the markers, it is able to yield relevant results with little effort.
We taint the pools and the stack, and scan copyout/copyoutstr. KLEAK is
supported on amd64 only for now, but it is not complicated to add more
architectures (just a matter of having the address of .text, and a stack
unwinder).
A userland tool is provided, that allows to execute a command in rounds
and monitor the leaks generated all the while.
KLEAK already detected directly 12 kernel info leaks, and prompted changes
that in total fixed 25+ leaks.
Based on an idea developed jointly with Thomas Barabosch (of Fraunhofer
FKIE).
g77 has likely not shipped as a binary in a long time.
No change for netbsd src as it has no fortran code, no change for pkgsrc
because it passes ${FC} for fortran packages.
Always define -fsanitize=${USE_SANITIZER} and
-fsanitize=${USE_LIBCSANITIZER} before other flags defined with
SANITIZERFLAGS and LIBCSANITIZERFLAGS.
This allows to overwrite rules that are parsed from left to right.
With the current approach it wasn't possible to disable some checks, as
they were reenabled again by the "-fsanitize=" argument.
Add flags that are required to build a program and a dynamically loaded
library.
Propagade LIBCSANITIZERFLAGS to CFLAGS and CXXFLAGS.
LDFLAGS is not changed on purpose, as a libcsanitizer is inside libc
and no external dependencies are needed.
Register NOLIBCSANITIZER, as certain distribution parts will need to be
skipped.
This change:
* Removes "options PERFCTRS", the associated includes, and the associated
ifdefs. In doing so, it removes several XXXSMPs in the MI code, which is
good.
* Removes the PMC code of ARM XSCALE.
* Removes all the pmc.h files. They were all empty, except for ARM XSCALE.
* Reorders the x86 PMC code not to rely on the legacy pmc.h file. The
definitions are put in sysarch.h.
* Removes the kern/sys_pmc.c file, and along with it, the sys_pmc_control
and sys_pmc_get_info syscalls. They are marked as OBSOL in kern,
netbsd32 and rump.
* Removes the pmc_evid_t and pmc_ctr_t types.
* Removes all the associated man pages. The sets are marked as obsolete.
support for only one ARM CPU. It used to have x86 support, but it was broken
and I removed it. The legacy PMC interface will be removed from the kernel
too. Sent on tech-kern@ yesterday, thorpej was fine.
A new keyword SANITIZER_RENAME_SYMBOL. It's a variable that has specified
a list of symbols to be renamed in a processor in order to remove naming
conflicts with public symbols in libc/m/pthread/rt.
This renaming will occur for libraries and programs.
A symbol with a conflicting 'name', will be renamed to '__mksanitizer_name'.
Based on an idea by <christos>
Note that the list of sanitizer features is just a selection, not a
complete list. Include there LSan and Scudo. Missing: hwmsan, esan,
ubsan_minimal etc.
Explain that USE_SANITIZER is an argument passed to -fsanitize= and it can
contain multiple options.
Mention SANITIZERFLAGS to pass even more sanitizer arguments.
Stop tracking in this file what compiler compiles what features. Just note
that the selection of supported features depends on a compiler version and
target CPU architecture.
Introduce a new variable SANITIZERFLAGS that contains all the sanitizer
specific flags. These flags are designed to be passed to CFLAGS, CXXFLAGS
and LDFLAGS.
Pass SANITIZERFLAGS in bsd.prog.mk rules to CFLAGS, CXXFLAGS and LDFLAGS.
Pass SANITIZERFLAGS in bsd.lib.mk rules to *_pic.a build flags. The *_pic.a
libraries are designed to construct .so libraries. All .so libraries with
few exceptions are expected to be prebuilt with a sanitizer. Another
purpuse of *_pic.a files can be constructing static PIE programs, however
they are not supported as of now and there is no sanitization of static
programs either.
Once MKSTATICPIE will be supported, it will be initially incompatible with
MKSANITIZER.
This change also makes it easier to build other targets like kernel and
kernel modules without userland sanitizers.
No functional change intended for MKSANITIZER=no users.
Allow to disable building groff. This is a build tool written in C++.
This option is useful with an external toolchain configured for building the
distribution rather than building tools (e.g. C++ lookup paths may differ).
Looks ok by <christos>
Sponsored by <The NetBSD Foundation>
use ${LD} ${LDFLAGS} -r which does not work if we specify MKRELRO since
LDFLAGS is supposed to be passed to LINK.c not LD directly (although some
of the flags work).
of /libdata/firmware and /usr/libdata/firmware are not installed
if set to "no". from the bsd.README:
MKFIRMWARE If "no", install the /libdata/firmware directory,
which is necessary for several drivers: athn(4), bwfm(4),
ipw(4), iwi(4), iwm(4), iwn(4), otus(4), rtwn(4), urtwn(4),
wpi(4), ral(4), rum(4), run(4), zyd(4), bcm43xx(4), and
the Tegra 124 SoC.
Default: yes on amd64, cobalt, evbarm evbmips, evbppc, hpcarm,
hppa, i386, mac68k, macppc, sandpoint, and sparc64, no elsewhere.
convert MKRADEONFIRMWARE and MKTEGRAFIRMWARE into FOO.arch lists,
and move their defaults above the general defaults setting.
saves about 12MB in a normal installation for other platforms.
programs there; make all Makefiles that use bsd.hostprog.mk include it.
Namely turn off MKREPRO and don't make lint, man pages, info files etc.
Remove the Makefile.inc files that contained these same settings, and
remove the settings from Makefile.host
- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
libtre cannot be used any more as a replacement for regex(3).
Tt does not build and the library development is stalled in upstream.
Proposed on mailing list.
Sponsored by <The NetBSD Foundation>
- remove old package makefiles
- update makekeys for Xfuncproto.h being a real file now
- clean up deleted package X11SRCDIR.pkg settings
this should be functionally equivalent.
this is the last GCC that will support these ports:
- epoc32
- netwinder
- shark
- acorn32
- cats
- most hpcarm systems (only NETBOOKPRO and WZERO3 remain)
- HAVE_GCC=5 is now the default (vs. HAVE_GCC=53 we've been using for
GCC 5.4 and GCC 5.5.)
- remove some more GCC 4.8 code. we don't support GCC 4 here.
- adjust set lists to gcc=5 from gcc=53.
add some basic HAVE_GCC=6 handling (totally unused so far.)
of coverity output when present [userland portion]
- --sysroot
- -fstack-protector
- -std=gnu99
- --nostdinc
- -fPIE
The critical one here being --sysroot because this means that we need to
use the actual system headers while building (or chroot/sandbox appropriately)
Provide a mechanism whereby a test sub-directory can be installed,
without the test being scheduled to run by default (ie: keeping
it out of the Atffile, and Kyuafile if Kyua is enabled.).
The mechanism is perhaps a bit kludgey - anyone with a better idea
how to make it happen, feel free to improve this (the one user as
of about the time of this commit is (or will be) src/tests/lib/Makefile)
position-independent, real PIC defaults to Global Dynamic as TLS model,
while non-PIC and PIE code can use more restrictive models like Initial
Exec. This is most visible with the thread_local destructor code now
using TLS in libc as it would be clobbered by any other shared library
with TLS due to static offset assignment by ld.
Originally, MKCRYPTO was introduced because the United States
classified cryptography as a munition and restricted its export. The
export controls were substantially relaxed fifteen years ago, and are
essentially irrelevant for software with published source code.
In the intervening time, nobody bothered to remove the option after
its motivation -- the US export restriction -- was eliminated. I'm
not aware of any other operating system that has a similar option; I
expect it is mainly out of apathy for churn that we still have it.
Today, cryptography is an essential part of modern computing -- you
can't use the internet responsibly without cryptography.
The position of the TNF board of directors is that TNF makes no
representation that MKCRYPTO=no satisfies any country's cryptography
regulations.
My personal position is that the availability of cryptography is a
basic human right; that any local laws restricting it to a privileged
few are fundamentally immoral; and that it is wrong for developers to
spend effort crippling cryptography to work around such laws.
As proposed on tech-crypto, tech-security, and tech-userlevel to no
objections:
https://mail-index.netbsd.org/tech-crypto/2017/05/06/msg000719.htmlhttps://mail-index.netbsd.org/tech-security/2017/05/06/msg000928.htmlhttps://mail-index.netbsd.org/tech-userlevel/2017/05/06/msg010547.html
P.S. Reviewing all the uses of MKCRYPTO in src revealed a lot of
*bad* crypto that was conditional on it, e.g. DES in telnet... That
should probably be removed too, but on the grounds that it is bad,
not on the grounds that it is (nominally) crypto.
If it's yes, all local symbols of shared libraries are stripped
(default). If it's no, only temporary local symbols are stripped;
for example, symbols of static functions are kept. Keeping such
symbols is useful on using DTrace for userland libraries and
getting a backtrace from a rump server loading modules (shared
libraries).
Proposed and discussed on tech-kern and tech-toolchain
restore ABI compatibility with previous releases for ieeefp.h on sh3.
add namespace.h protection for all the fenv interfaces.
use MKSOFTFLOAT on sh3 instead of assuming softfloat.
standardize on comparing MKSOFTFLOAT with "no".
remove the arm-specific softfloat fenv code (which also had several bugs).
fix logic errors in the arm hardfloat feraiseexcept() and feupdateenv().
libpthread_dbg(3) is a remnant library from the M:N thread model
(pre-NetBSD-5.0) API to introspect threads within a process and for use
of debuggers.
Currently in the 1:1 model it's not used in GDB neither in LLDB and it's
not either planned to be used. It's current function to read pthread_t
structures is realizable within a regular debugger capable to
instrospect objects within a tracee (GDB, LLDB...).
Remaining users of this API can still use this library from
pkgsrc/devel/libpthread_dbg.
Sponsored by <The NetBSD Foundation>
maxv