code is fast, with an nonexistent overhead - and we already take care of
setting MT_FREE, so why not check it.
In addition, stop registering the function name, that's not helpful since
the MBUFFREE macro is local. Instead, set m_data to NULL, so that any
access to a freed mbuf's data after mtod() or similar will page fault.
The combination of these two changes provides a fast and efficient way of
detecting use-after-frees in the network stack.
The caller must ensure that no new work is enqueued before calling
workqueue_wait. Note that Note that if the workqueue is WQ_PERCPU, the caller
can enqueue a new work to another queue other than the waiting queue.
Discussed on tech-kern@
sys_pipe2() returns two integers (values), the 2nd one is a copy of the 2nd
file descriptor that lands in fildes[2]. This is a side effect of reusing
the code for sys_pipe() (SYS_pipe) and not cleaning it up.
The first returned value is (on success) 0.
Introduced a small refactoring in pipe1() that it does not operate over
retval[], but on an array int[2]. A user sets retval[] for pipe() when
desired and needed.
This refactoring touches compat code: netbsd32, linux, linux32.
Before the changes on NetBSD/amd64:
$ ktruss -i ./a.out
[...]
15131 1 a.out pipe2(0x7f7fff2e62b8, 0) = 0, 4
[...]
After the changes:
$ ktruss -i ./a.out
[...]
782 1 a.out pipe2(0x7f7fff97e850, 0) = 0
[...]
There should not be a visible change for current users.
Sponsored by <The NetBSD Foundation>
sstk - change stack section size
This functionality has never been implemented and is a remnant from 16-bit
UNIX. This stub appeared with the first NetBSD commit.
Sponsored by <The NetBSD Foundation>
if a pool is growing, and the grower is PR_NOWAIT, mark this.
if another caller wants to grow the pool and is also PR_NOWAIT,
busy-wait for the original caller, which should either succeed
or hard-fail fairly quickly.
implement the busy-wait by unlocking and relocking this pools
mutex and returning ERESTART. other methods (such as having
the caller do this) were significantly more code and this hack
is fairly localised.
ok chs@ riastradh@
doesn't help. The check really cannot fail, and it only looks at the list
belonging to the current level of recursion.
Instead, verify that the module's modcmd(MODULE_CMD_INIT, ...) does not
introduce a duplicate module name as a result of recursively calling
module_do_load().
- Make psref_check_duplication check just if a given psref is on the list
- It checked both psref and target
- Suggested by riastradh@ some time ago
- Add psref_check_existence that checks a releasing psref is surely on the list
with the same internal name but no conflicting symbol definitions.
When we load a module from the file system, the filename may have no
relationship to the internal module's name. Furthermore, comparing
the module's filename is insufficient if the file is loaded from an
absolute path.
XXX pullup to netbsd-8
for each transfer to write through to the medium is way slower than just
letting the drive use a cached write and doing DIOCCACHESYNC on the end
Results were (fs block 32KB / frag 4KB, partition aligned on 32KB boundary):
HDD at siisata(4): no-FUA: 108 sec w/FUA: 294 sec
SSD at ahcisata(4): no-FUA: 73 sec w/FUA: 502 sec
change the flag so that FUA is only used for the commit block write;
for journal data write, only pass DPO, rely on the cache flush to get them
to media
bootpages is set to the pages allocated via uvm_pageboot_alloc().
poolpages is calculated from the list of pools nr_pages members.
this brings us closer to having a valid total of pages known by
the system, vs actual pages originally managed.
XXX: poolpages needs some handling for PR_RECURSIVE pools still.
typing 'cat /proc/aout_pid/auxv' on whatever a.out binary you're running.
Fortunately, amd64 does not enable EXEC_AOUT by default. Unfortunately,
i386 does enable it by default.
(and now) different kqueue behavior between NetBSD and other kqueue
implementations that depend on specific file types. If 3rd party programs
trigger this it is probably because we are doing something different.
If IFEF_MPSAFE is set, hold the lock and otherwise don't hold.
This change requires additions of KERNEL_LOCK to subsequence functions from
if_ioctl such as ifmedia_ioctl and ifioctl_common to protect non-MP-safe
components.
Proposed on tech-kern@ and tech-net@
some device drivers and some pseudo interfaces allocate a lot of softints. The
resource size for softints are static and it panics when it execeed the limit.
It can be dynamically resized. Untill dynamically resizing is implemented,
increase softint_bytes from 8192 to 32768.
The debugging feature detects violations of pserialize constraints.
It causes a panic:
- if a context switch happens in a read section, or
- if a sleepable function is called in a read section.
The feature is enabled only if LOCKDEBUG is on.
Discussed on tech-kern@
We cannot get an accurate count from a localcount instance because it consists
of per-cpu counters and we have no way to sum them up atomically. So we cannot
detect counter overflow/underflow as we can do on a normal refcount.
The facility adds an atomic counter to each localcount instance to enable the
validations. The counter ups and downs in synchronization with the per-CPU
counters. The counter is used iff both DEBUG and LOCKDEBUG are enabled in the
kernel.
Discussed on tech-kern@
- when priming the pool and failing with ERESTART, don't decrement the number
of pages; this avoids the issue of returning an ERESTART when we get to 0,
and is more correct.
- simplify the pool_grow code, and don't wakeup things if we ENOMEM.
resolved pathname. We need this in the case of scripts where p_path needs
to point to the interpreter and not the script itself. Otherwise things
like perl script that depend on /proc/$$/exe to re-exec themselves end up
being fork bombs.
In reality we should be using the fully resolved/canonicalized path here, but
namei is not giving it back to us.
This means that the full executable path is always available.
- exec_elf.c: use p->path to set AT_SUN_EXECNAME, and since this is
always set, do so unconditionally.
- kern_exec.c: simplify pathexec, use kmem_strfree where appropriate
and set p->p_path
- kern_exit.c: free p->p_path
- kern_fork.c: set p->p_path for the child.
- kern_proc.c: use p->p_path to return the executable pathname; the
NULL check for p->p_path, should be a KASSERT?
- exec.h: gc ep_path, it is not used anymore
- param.h: bump version, 'struct proc' size change
TODO:
1. reference count the path string, to save copy at fork and free
just before exec?
2. canonicalize the pathname by changing namei() to LOCKPARENT
vnode and then using getcwd() on the parent directory?
This would have caught the mistake I made last week leading to null
pointer dereferences all over the place, a mistake which I evidently
poorly scheduled alongside maxv's change to the panic message on x86
for null pointer dereferences.
the kernel and module symbols, and when relocating a symbol that has
SHN_ABS, take its value as-is and don't return an error if it equals zero.
Sent on tech-kern@.
PROC_MACHINE_ARCH32(P) to override the value for sysctl hw.machine_arch
(native and netbsd32 commpat resp.).
Use these for arm and mips instead of the (not working, noisy, in case
of arm) sysctl override and #ifdef __mips__ in architecture neutral
code.
Candidate fix for problems with hanging after kva fragmentation related
to PR kern/45718.
Proposed on tech-kern:
https://mail-index.NetBSD.org/tech-kern/2017/10/23/msg022472.html
Tested by bouyer@ on i386.
This makes one small change to the semantics of pool_prime and
pool_setlowat: they may fail with EWOULDBLOCK instead of ENOMEM, if
there is a pending call to the backing allocator in another thread but
we are not actually out of memory. That is unlikely because nearly
always these are used during initialization, when the pool is not in
use.
XXX pullup-8
XXX pullup-7
XXX pullup-6 (requires tweaking the patch)
XXX pullup-5...
in PR kern/52639, as well as some general cleaning-up...
(As proposed on tech-kern@ with additional changes and enhancements.)
Details of changes:
* All history arguments are now stored as uintmax_t values[1], both in
the kernel and in the structures used for exporting the history data
to userland via sysctl(9). This avoids problems on some architectures
where passing a 64-bit (or larger) value to printf(3) can cause it to
process the value as multiple arguments. (This can be particularly
problematic when printf()'s format string is not a literal, since in
that case the compiler cannot know how large each argument should be.)
* Update the data structures used for exporting kernel history data to
include a version number as well as the length of history arguments.
* All [2] existing users of kernhist(9) have had their format strings
updated. Each format specifier now includes an explicit length
modifier 'j' to refer to numeric values of the size of uintmax_t.
* All [2] existing users of kernhist(9) have had their format strings
updated to replace uses of "%p" with "%#jx", and the pointer
arguments are now cast to (uintptr_t) before being subsequently cast
to (uintmax_t). This is needed to avoid compiler warnings about
casting "pointer to integer of a different size."
* All [2] existing users of kernhist(9) have had instances of "%s" or
"%c" format strings replaced with numeric formats; several instances
of mis-match between format string and argument list have been fixed.
* vmstat(1) has been modified to handle the new size of arguments in the
history data as exported by sysctl(9).
* vmstat(1) now provides a warning message if the history requested with
the -u option does not exist (previously, this condition was silently
ignored, with only a single blank line being printed).
* vmstat(1) now checks the version and argument length included in the
data exported via sysctl(9) and exits if they do not match the values
with which vmstat was built.
* The kernhist(9) man-page has been updated to note the additional
requirements imposed on the format strings, along with several other
minor changes and enhancements.
[1] It would have been possible to use an explicit length (for example,
uint64_t) for the history arguments. But that would require another
"rototill" of all the users in the future when we add support for an
architecture that supports a larger size. Also, the printf(3) format
specifiers for explicitly-sized values, such as "%"PRIu64, are much
more verbose (and less aesthetically appealing, IMHO) than simply
using "%ju".
[2] I've tried very hard to find "all [the] existing users of kernhist(9)"
but it is possible that I've missed some of them. I would be glad to
update any stragglers that anyone identifies.
vmin is only an optional hint since we're not passing UVM_FLAG_FIXED,
but that doesn't mean we should use uninitialized stack garbage as
the hint.
Noted by chs@.
Candidate fix for PR kern/45718: `processes sometimes get stuck and
spin in vm_map', a problem that has been plaguing all our 32-bit
ports for years.
Since we currently use large (256k) buffers for execargs, and since
nobody has stepped up to tackle breaking them into bite-sized (or at
least page-sized) chunks, after KVA gets sufficiently fragmented we
can't allocate new execargs buffers from kernel_map.
Until 2008, we always carved out KVA for execargs on boot with a uvm
submap exec_map of kernel_map. Then ad@ found that the uvm_km_free
call, to discard them when done, cost about 100us, which a pool
avoided:
https://mail-index.NetBSD.org/tech-kern/2008/06/25/msg001854.htmlhttps://mail-index.NetBSD.org/tech-kern/2008/06/26/msg001859.html
ad@ _simultaneously_ introduced a pool _and_ eliminated the reserved
KVA in the exec_map submap. This change preserves the pool, but
restores exec_map (with less code, by putting it in MI code instead
of copying it in every MD initialization routine).
Patch proposed on tech-kern:
https://mail-index.NetBSD.org/tech-kern/2017/10/19/msg022461.html
Patch tested by bouyer@:
https://mail-index.NetBSD.org/tech-kern/2017/10/20/msg022465.html
I previously discussed the issue on tech-kern before I knew of the
history around exec_map:
https://mail-index.NetBSD.org/tech-kern/2012/12/09/msg014695.html
The candidate workaround I proposed of using pool_setlowat to force
preallocation of KVA would also force preallocation of physical RAM,
which is a waste not incurred by using exec_map, and which is part of
why I never committed it.
There may remain a general problem that if thread A calls pool_get
and tries to service that request by a uvm_km_alloc call that hangs
because KVA is scarce, and thread B does pool_put, the pool_put in
thread B will not notify the pool_get in thread A that it doesn't
need to wait for KVA, and so thread A may continue to hang in
uvm_km_alloc. However,
(a) That won't apply here, because there is exactly as much KVA
available in exec_map as exec_pool will ever try to use.
(b) It is possible that may not even matter in other cases as long as
the page daemon eventually tries to shrink the pool, which will cause
a uvm_km_free that can unhang the hung uvm_km_alloc.
XXX pullup-8
XXX pullup-7
XXX pullup-6
XXX pullup-5, perhaps...
There is a race here, as seen on arm with FPU:
LWP L is running but not on CPU, has its FPU state on CPU2 which
has not been released yet, so fpexc still has VFP_FPEXC_EN set in the PCB copy.
LWP L is scheduled on CPU1, CPU1 calls cpu_switchto() for L in mi_switch().
cpu_switchto() will set VFP_FPEXC_EN in the FPU's fpexc register per the
PCB fpexc copy.
Before CPU1 calls pcu_switchpoint() for L, CPU2 calls
pcu_do_op(PCU_CMD_SAVE | PCU_CMD_RELEASE) for L because it still holds its
FPU state and wants to load another lwp. This cause VFP_FPEXC_EN to
be cleared in the PCB copy, but not in CPU1's register. L's l_pcu_cpu is
set to NULL.
When CPU1 calls pcu_switchpoint() for L it see l_pcu_cpu is NULL, and doesn't
call the release callback.
Now CPU1 has its FPU enabled but with the wrong FPU state.
Fix by releasing the PCU even if l_pcu_cpu is NULL.
disable it - which defaults to disabled. The following command is now
required to use linux binaries:
sysctl -w emul.linux.enabled=1
After a discussion on tech-kern@. All the other ideas to reduce the attack
surface have drawbacks, and this sysctl seems to be the best option.
handling into the backend and doing an optimistic (unlocked) check
first. Always taking the vnode interlock makes this assertion otherwise
very heavy for multi-processor machines. Ride the kernel version bump.
all the same values (except for the filename) just ignore it. Otherwise
report the duplicate-entry error.
This allows the user to create a signature file with veriexegen(8) and
not worry about duplicate entries (due to hard-linked files) which will
otherwise cause /etc/rc.d/veriexec to report an error.
Fixes PR kern/52512
XXX Pull-up for -8
This is a legacy interface from 4.4BSD, and it was
introduced to overcome shortcomings of ptrace(2) at that time, which are
no longer relevant (performance). Today /proc/#/ctl offers a narrow
subset of ptrace(2) commands and is not applicable for modern
applications use beyond simplistic tracing scenarios.
This removal will simplify kernel internals. Users will still be able to
use all the other /proc files.
This change won't affect other procfs files neither Linux compat
features within mount_procfs(8). /proc/#/ctl isn't available on Linux.
Remove:
- /proc/#/ctl from mount_procfs(8)
- P_FSTRACE note from the documentation of ps(1)
- /proc/#/ctl and filesystem tracing documentation from mount_procfs(8)
- KAUTH_REQ_PROCESS_PROCFS_CTL documentation from kauth(9)
- source code file miscfs/procfs/procfs_ctl.c
- PFSctl and procfs_doctl() from sys/miscfs/procfs/procfs.h
- KAUTH_REQ_PROCESS_PROCFS_CTL from sys/sys/kauth.h
- PSL_FSTRACE (0x00010000) from sys/sys/proc.h
- P_FSTRACE (0x00010000) from sys/sys/sysctl.h
Reduce code complexity after removal of this functionality.
Update TODO.ptrace accordingly: remove two entries about /proc tracing.
Do not keep legacy notes as comments in the headers about removed
PSL_FSTRACE / P_FSTRACE, as this interface had little number of users
(close or equal to zero).
Proposed on tech-kern@.
All filesystem tracing utility users are encouraged to switch to ptrace(2).
Sponsored by <The NetBSD Foundation>
of this file only.
Rather than adding meaningless {} around all uses of functions that
are #defined to nothing for userland, #define the funcs to something
that is functionally equivalent (but which appeases gcc).
Also, define KASSERT() to nothing for userland, which avoids the need
to add a #definee for mutex_owned which would otherwise be needed,
and simmultaneoiusly stops gcc from complaining about a lack of a prototype.
autoloaded modules. These options are disabled everywhere (except ibcs2
on Vax, but Vax does not support kernel modules, so doesn't matter),
therefore there is no issue in removing them from the list. Interested
users will now have to do a 'modload' first, or uncomment the entries in
GENERIC.
will now have to type 'modload' to use it, or uncomment the entry in
GENERIC. I should have removed it when I disabled COMPAT_FREEBSD by
default, sorry about that.
Takes struct bintime maximum delay, and decrements it in place so
that you can use it in a loop in case of spurious wakeup.
Discussed on tech-kern a couple years ago:
https://mail-index.netbsd.org/tech-kern/2015/03/23/msg018557.html
Added a parameter for expressing desired precision -- not currently
interpreted, but intended for a future tickless kernel with a choice
of high-resolution timers.
complementary writable sysctl for the initial guard size of threads
created via pthread_create. Let the existing attribut accessors do the
right thing. Raise the default guard size for threads to 64KB.
Always include a 1MB guard area beyond the end of stack. While ASLR will
normally create a guard area as well, this provides a deterministic area
for all binaries.
Mitigates the rest of CVE-2017-1000374 and CVE-2017-1000375 from
Qualys.
Additionally, change VM_DEFAULT_ADDRESS_TOPDOWN to include
user_stack_guard_size in the size reservation.
Don't check for negative; it does not matter we clamp anyway. This
broke the compat32 getsockname() where an unitialized socklen_t ended
up randomly negative causing it to fail.
normally create a guard area as well, this provides a deterministic area
for all binaries.
Mitigates the rest of CVE-2017-1000374 and CVE-2017-1000375 from
Qualys.
this is needed to avoid name conflicts with ZFS and also
makes it clearer that other code shouldn't be messing with these.
remove the LFS debug code that poked around in bufqueues and
remove the BQ_EMPTY bufqueue since nothing uses it anymore.
provide a function to let LFS and wapbl read the value of nbuf for now.
this sequence is used by ZFS in a couple places and by supporting it
natively we can undo our local ZFS changes that avoided it.
note that this is only legal when all of the waiters use cv_wait()
and not any of the other variations, and lockdebug will catch
any violations of this rule.
use FSTRANS_SHARED as lock type so remove the lock type argument.
File system state FSTRANS_SUSPENDING is now unused so remove it.
Regen vnode_if files.
Ride 8.99.1 less than a hour ago.
Add two "static inline" functions to vnode_if.c to handle MPSAFE
and FSTRANS before and after the "VCALL()".
Take FSTRANS and handle error before "VCALL(...vop_lock...)" and
release it after "VCALL(...vop_unlock...)".
node and a usecount greater zero. Therefore rename state "VS_ACTIVE"
to "VS_LOADED" and add a new synthetic state "VS_ACTIVE" for VSTATE_ASSERT()
to assert an active vnode.
Add VSTATE_ASSERT_UNLOCKED() to be used with v_interlock unheld and
move the state assertion macros to sys/vnode_impl.h.
kmem_alloc() with KM_SLEEP
kmem_zalloc() with KM_SLEEP
percpu_alloc()
pserialize_create()
psref_class_create()
all of these paths include an assertion that the allocation has not failed,
so callers should not assert that again.
CAN stands for Controller Area Network, a broadcast network used
in automation and automotive fields. For example, the NMEA2000 standard
developped for marine devices uses a CAN network as the link layer.
This is an implementation of the linux socketcan API:
https://www.kernel.org/doc/Documentation/networking/can.txt
you can also see can(4).
This adds a new socket family (AF_CAN) and protocol (PF_CAN),
as well as the canconfig(8) utility, used to set timing parameter of
CAN hardware. Also inclued is a driver for the CAN controller
found in the allwinner A20 SoC (I tested it with an Olimex lime2 board,
connected with PIC18-based CAN devices).
There is also the canloop(4) pseudo-device, which allows to use
the socketcan API without CAN hardware.
At this time the CANFD part of the linux socketcan API is not implemented.
Error frames are not implemented either. But I could get the cansend and
canreceive utilities from the canutils package to build and run with minimal
changes. tcpudmp(8) can also be used to record frames, which can be
decoded with etherreal.
VOP_RECLAIM naturally has exclusive access to the vnode, so having it
locked on entry is not strictly necessary -- but it means if there
are any final operations that must be done on the vnode, such as
ffs_update, requiring exclusive access to it, we can now kassert that
the vnode is locked in those operations.
We can't just have the caller release the last lock because some file
systems don't use genfs_lock, and require the vnode to remain valid
for VOP_UNLOCK to work, notably unionfs.
use with mprotect(2), but without enabling them immediately.
Extend the mremap(2) interface to allow duplicating mappings, i.e.
create a second range of virtual addresses references the same physical
pages. Duplicated mappings can have different effective protections.
Adjust PAX mprotect logic to disallow effective protections of W&X, but
allow one mapping W and another X protections. This obsoletes using
temporary files for purposes like JIT.
Adjust PAX logic for mmap(2) and mprotect(2) to fail if W&X is requested
and not silently drop the X protection.
Improve test cases to ensure correct operation of the changed
interfaces.
PT_SETSTEP and PT_CLEARSTEP in the current design must unlock proc_lock and
t->p_lock. These functions use lwp_delref() for a tracee with more than one
LWP. This function internally lock (t->)p_lock and this is lock against
self.
There are coming new ATF test with PT_*STEP with multiple LWPs to catch
these bugs in future changes.
Sponsored by <The NetBSD Foundation>
at the place we expected it to be attached!
As mentioned several times (on tech-kern@ mailing list) over the past
18 months or so, I've seen a few instances where this will trigger,
although I've been unable to reproduce them. Hopefully some wider
exposure will reveal the under-lying cause of this rare phenomenon.
Commit was proposed on tech-kern list, and no objections raised.
lwp_create() has been acquired more arguments, there was missing the latest
one. Per analogiam with changes in the same commit to other source files,
go for &SS_INIT.
Print e_ident[EI_MAG3] (it was missed)
Print e_ident[EI_CLASS] as it is used do determine correct ELF magic.
No functional change for non-debug (without option DEBUG_ELF) build.
- remove now unused field mnt_list.
- rename mount_list to mountlist and make it local to vfs_mount.c.
- make mountlist_lock local to vfs_mount.c.
Change pstat.c to retrieve vnodes by lru lists.
Don't count arguments that have WILLRELE/WILLPUT; count arguments
that are struct vnode *.
No functional change currently because it happens that every released
or put vnode argument comes first or after other ones.
Breaks file systems for which VOP_UNLOCK doesn't work on a reclaimed
vnode.
The only case in tree right now is sys/fs/union -- most file systems
use genfs_unlock, which does work on a reclaimed vnode.
Maybe we can work around this -- and still enable VOP_RECLAIM's
callees to assert lock ownership -- by having VOP_RECLAIM unlock the
vnode instead.
No bump because it wouldn't have been possible to acquire the lock in
VOP_RECLAIM anyway -- instant deadlock because vn_lock waits to
transition out of the RECLAIMING state first. Benefit is that we can
now assert ownership of the lock in any operations called by
VOP_RECLAIM.
Discussed on tech-kern:
https://mail-index.netbsd.org/tech-kern/2017/04/01/msg021751.html
by default, add sysctl vfs.wapbl.journal_iobufs to control it
this also removes need to allocate iobuf during commit, so it
might help to avoid deadlock during memory shortages like PR kern/47030
These operations allow to mark thread as a single-stepping one.
This allows to i.a.:
- single step and emit a signal (PT_SETSTEP & PT_CONTINUE)
- single step and trace syscall entry and exit (PT_SETSTEP & PT_SYSCALL)
The former is useful for debuggers like GDB or LLDB. The latter can be used
to singlestep a usermode kernel. These examples don't limit use-cases of
this interface.
Define PT_*STEP only for platforms defining PT_STEP.
Add new ATF tests setstep[1234].
These ptrace(2) operations first appeared in FreeBSD.
Sponsored by <The NetBSD Foundation>
calling the operation on the lower vnode.
Replace vi_lock with a rw_obj and change layered file systems
to share the lock with the lower vnode.
Layered file systems now use genfs_lock()/_unlock/_islocked().
Welcome to 7.99.67
Pointed out by Christous Zoulas that ELF_AUX_ENTRIES * sizeof(AuxInfo)
assumption is incomplete. There is emulation code that can use different
values (smaller and larger).
