allowing -lrump to be used without -lrumpvfs.
This is an alternate fix to the earluer one which added -lrumvfs to
many rump based tests (and the rump server) which might be undone soon.
This also fixes the sun2 build.
It is yet another psref leak detector that enables to tell where a leak occurs
while a simpler version that is already committed just tells an occurrence of a
leak.
Investigating of psref leaks is hard because once a leak occurs a percpu list of
psref that tracks references can be corrupted. A reference to a tracking object
is memorized in the list via an intermediate object (struct psref) that is
normally allocated on a stack of a thread. Thus, the intermediate object can be
overwritten on a leak resulting in corruption of the list.
The tracker makes a shadow entry to an intermediate object and stores some hints
into it (currently it's a caller address of psref_acquire). We can detect a
leak by checking the entries on certain points where any references should be
released such as the return point of syscalls and the end of each softint
handler.
The feature is expensive and enabled only if the kernel is built with
PSREF_DEBUG.
Proposed on tech-kern
rumpdev to rumpkern, liberating all rumpnet users from the need to
-lrumpdev -lrumpvfs just because a loopback interface is mandatory.
Rename rumpdev/autoconf.c to rumpkern/rump_autoconf.c to avoid
accidentally picking up e.g. sys/arch/amd64/amd64/autoconf.c through
make's .PATH.
Move rumpdev/MAINBUS.ioconf to rumpkern.
- Interrupt-oriented system rather than thread-oriented.
- Improve stability, quality and performance.
- Split playback and record cleanly. Improve halfduplex support.
- Many bugs are fixed including deadlocks, resource leaks, abuses, etc.
- Simplify audio filter mechanism. The encoding/channels/frequency
conversions are completely handled in the upper layer. So the hard-
ware driver only converts its hardware encoding (if necessary).
- audio_hw_if changes:
- Obsoletes query_encoding and add query_format instead.
- Obsoletes set_params and add set_format instead.
- Remove drain, setfd, mappage.
- The call sequences are changed.
- ioctl AUDIO_GETFD/SETFD, AUDIO_GETCHAN/SETCHAN are obsoleted.
- ioctl AUDIO_{QUERY,GET,SET}FORMAT are introduced.
- cleanup config attributes: au*conv and mulaw.
- All hardware drivers should follow it (I've done as much as possible).
Some file paths are changed:
- dev/audio.c -> dev/audio/audio.c (rewritten)
- dev/audiovar.h -> dev/audio/audiovar.h
- dev/audio_dai.h -> dev/audio/audio_dai.h
- dev/audio_if.h -> dev/audio/audio_if.h
- dev/audiobell.c -> dev/audio/audiobell.c
- dev/audiobellvar.h -> dev/audio/audiobellvar.h
- dev/mulaw.[ch] -> dev/audio/mulaw.[ch] + dev/audio/alaw.c
- Defer spa_config_load() until root is mounted.
- Restore the config path to "/etc/zfs/zpool.cache".
- Module "zfs" is type MODULE_CLASS_VFS and no longer depends on "rootvnode".
- Module "solaris" no longer depends on "mp_online".
- Fix rump component registration to not detach "/dev/zfs" if
it didn't attach it.
It detects leaks by counting up the number of held psref by an LWP and checking
its zeroness at the end of syscalls and softint handlers. For the counter, a
unused field of struct lwp is reused.
The detector runs only if DIAGNOSTIC is turned on.
userspace. The old fetch(9) and store(9) APIs (fubyte(), fuword(),
subyte(), suword(), etc.) are retired and replaced with new ufetch(9)
and ustore(9) APIs that can return proper error codes, etc. and are
implemented consistently across all platforms. The interrupt-safe
variants are no longer supported (and several of the existing attempts
at fuswintr(), etc. were buggy and not actually interrupt-safe).
Also augmement the ucas(9) API, making it consistently available on
all plaforms, supporting uniprocessor and multiprocessor systems, even
those that do not have CAS or LL/SC primitives.
Welcome to NetBSD 8.99.37.
valid "mnt_transinfo" and remove now unneeded flag IMNT_HAS_TRANS.
Run fstrans_start()/fstrans_done() on dead_rootmount if FSTRANS_DEAD_ENABLED.
Should become the default for DIAGNOSTIC in the future.
this is not true for alpha, ia64 and arm32 ports, and the first two
were not building because of it, and the latter would be missing
the oabi support (likely not a big deal, but still wrong.)
add a makefile fragment that tells you if it is supported and include
it where needed to define COMPAT_NETBSD32 when building the normal
kernel (ie, modules & rump.)
fixes alpha build, probably fixes ia64 build.
XXX: still leaves some netbsd32 code in rf_netbsdkintf.c, that should
be moved into some hooks, but first the configuration setup
needs to be moved into a common function the netbsd32 code can
call into, vs living in the switch case itself.
XXX arm_icache_sync_range() and mips_icache_sync_range() call
(void)rumpcomp_sync_icache((void *)va, (uint64_t)sz);
but linking fails if I do the same on aarch64 (I suspect it also
fails on 32bit arm and mips).
As a workaround, I call __builtin___clear_cache().
threads running at specific priorities, with support for unbound pools
and per-cpu pools.
Written by riastradh@, and based on the May 2014 draft, with a few changes
by me:
- Working on the assumption that a relative few priorities will actually
be used, reduce the memory footprint by using linked lists, rather than
2 large (and mostly empty) tables. The performance impact is essentially
nil, since these lists are consulted only when pools are created (and
destroyed, for DIAGNOSTIC checks), and the lists will have at most 225
entries.
- Make threadpool job object, which the caller must allocate storage for,
really opaque.
- Use typedefs for the threadpool types, to reduce the verbosity of the
API somewhat.
- Fix a bunch of pool / worker thread / job object lifecycle bugs.
Also include an ATF unit test, written by me, that exercises the basics
of the API by loading a kernel module that exposes several sysctls that
allow the ATF test script to create and destroy threadpools, schedule a
basic job, and verify that it ran.
And thus NetBSD 8.99.29 has arrived.
hashing and radix trie. It supports lock-free lookups and concurrent
inserts/deletes. It is designed to be optimal as a general purpose
*concurrent* associative array.
Upstream: https://github.com/rmind/thmap
Discussed on tech-kern@
software by ether_sw_offload_[tr]x().
For rump kernels, if_capabilities for shmemif(4) can be specified
by environmental variable RUMP_SHMIF_CAPENABLE:
setenv RUMP_SHMIF_CAPENABLE 0x7ff80 (all offload)
setenv RUMP_SHMIF_CAPENABLE 0x6aa80 (all TX)
setenv RUMP_SHMIF_CAPENABLE 0x15500 (all RX)
part of PR kern/53562
OK msaitoh
Handle TX offload in software when a packet is sent via
bridge_output(). We can send it as is in the following
exceptional cases:
For unicast:
(1) When the destination interface is the same as source.
(2) When the destination supports all TX offload options
specified in a packet.
For multicast/broadcast:
(3) When all the members of the bridge support the specified
TX offload options.
For (3), add sc_csum_flags_tx flag to bridge softc, which is
logical AND b/w capabilities of TX offload options in member
interface (ifp->if_csum_flags_tx). The flag is updated when a
member is (i) added to or (ii) removed from a bridge, or (iii)
if_csum_flags_tx flag of a member interface is manipulated via
ifconfig(8).
Turn on M_CSUM_TSOv[46] bit in ifp->if_csum_flags_tx flag when
TSO[46] is enabled for that interface.
OK msaitoh thorpej
Add ether_sw_offload_[tr]x: handle TX/RX offload options in software.
Since this violates separation b/w L2 and L3/L4, new files are added
rather than having the routines in sys/net/if_ethersubr.c.
OK msaitoh thorpej
(Only for architectures that support it at all; on others,
__HAVE_MM_MD_DIRECT_MAPPED_PHYS/IO are still undefined and the
functions unimplemented.)
This gives modules like zfs an opportunity to use it.
While here, fix the one caller of mm_md_direct_mapped_phys that
ignored the return value (and make sure to call pmap_kremove/update
before uvm_km_free).
NORMAL -> SUSPENDING -> SUSPENDED
and add operation fstrans_start_lazy() that only blocks while SUSPENDED.
Change vndthread() support operation handle_with_rdwr() to bracket
its file system operations by fstrans_start_lazy() and fstrans_done().
PR kern/53624 (dom0 freeze on domU exit)
- This conversion significantly simplifies the code and moves NPF to
a binary serialisation format (replacing the XML-like format).
- Fix some memory/reference leaks and possibly use-after-free bugs.
- Bump NPF_VERSION as this change makes libnpf incompatible with the
previous versions. Also, different serialisation format means NPF
connection/config saving and loading is not compatible with the
previous versions either.
Thanks to christos@ for extra testing.
These functions are defined on unsigned int. The generic name
min/max should not silently truncate to 32 bits on 64-bit systems.
This is purely a name change -- no functional change intended.
HOWEVER! Some subsystems have
#define min(a, b) ((a) < (b) ? (a) : (b))
#define max(a, b) ((a) > (b) ? (a) : (b))
even though our standard name for that is MIN/MAX. Although these
may invite multiple evaluation bugs, these do _not_ cause integer
truncation.
To avoid `fixing' these cases, I first changed the name in libkern,
and then compile-tested every file where min/max occurred in order to
confirm that it failed -- and thus confirm that nothing shadowed
min/max -- before changing it.
I have left a handful of bootloaders that are too annoying to
compile-test, and some dead code:
cobalt ews4800mips hp300 hppa ia64 luna68k vax
acorn32/if_ie.c (not included in any kernels)
macppc/if_gm.c (superseded by gem(4))
It should be easy to fix the fallout once identified -- this way of
doing things fails safe, and the goal here, after all, is to _avoid_
silent integer truncations, not introduce them.
Maybe one day we can reintroduce min/max as type-generic things that
never silently truncate. But we should avoid doing that for a while,
so that existing code has a chance to be detected by the compiler for
conversion to uimin/uimax without changing the semantics until we can
properly audit it all. (Who knows, maybe in some cases integer
truncation is actually intended!)
entry-point is either sys_nomodule or sys_nosys. Update the
makesyscalls.sh script to create a const array of bits to allow
syscall_disestablish() to properly restore the original entry-point.
Update all the initializers of struct emul to initialize the pointer
to the bit array struct emul.
XXX Regen of all files created by makesyscalls.sh will come soon,
XXX followed by a kernel version bump (since struct emul is being
XXX modified).
This commit should address PR kern/45781 and also removes the need
for the work-around for that PR in file
sys/arch/usermode/modules/syscallemu/syscallemu.c
This change:
* Removes "options PERFCTRS", the associated includes, and the associated
ifdefs. In doing so, it removes several XXXSMPs in the MI code, which is
good.
* Removes the PMC code of ARM XSCALE.
* Removes all the pmc.h files. They were all empty, except for ARM XSCALE.
* Reorders the x86 PMC code not to rely on the legacy pmc.h file. The
definitions are put in sysarch.h.
* Removes the kern/sys_pmc.c file, and along with it, the sys_pmc_control
and sys_pmc_get_info syscalls. They are marked as OBSOL in kern,
netbsd32 and rump.
* Removes the pmc_evid_t and pmc_ctr_t types.
* Removes all the associated man pages. The sets are marked as obsolete.
The node (and child nodes) is initialized in sysctl_net_pktq_setup, but the call
of sysctl_net_pktq_setup is skipped unexpectedly.
sysctl_net_pktq_setup is skipped if in6_present is false that indicates the
netinet6 component isn't loaded on rump kernels. However the flag is
accidentally always false because the flag is turned on in in6_dom_init that is
called after if_sysctl_setup on both normal and rump kernels.
Fix the issue by moving if_sysctl_setup after in6_dom_init (domaininit on normal
kernels). This fix is ad-hoc but good enough for netbsd-8. We should refine
the initialization order of network components in the future.
Pointed out by hikaru@
compatibility with BIOC[GS]SEESENT ioctl. The userland interface is the same
as FreeBSD.
This change also fixes a bug that the direction is misunderstand on some
environment by passing the direction to bpf_mtap*() instead of checking
m->m_pkthdr.rcvif.
this changes the upstream vendor from OpenSolaris to FreeBSD,
and this version is based on FreeBSD svn r315983.
in addition to the 10 years of improvements from upstream,
this version also has these NetBSD-specific enhancements:
- dtrace FBT probes can now be placed in kernel modules.
- ZFS now supports mmap().
a bool for clarity. Optimize the function: if M_CANFASTFWD is not there
(because already removed by the firewall) leave now.
Makes it easier to see that M_CANFASTFWD is not removed on IPv6.
- sys/arch/evbarm64 is gone and integrated into sys/arch/evbarm. (by skrll@)
- add support fdt. evbarm/conf/GENERIC64 fdt (bcm2837,sunxi,tegra) based generic 64bit kernel config. (by skrll@, jmcneill@)
A deadlock occurs because there is a violation of the rule of lock ordering;
softnet_lock is held with hodling IFNET_LOCK, which violates the rule.
To avoid the deadlock, replace softnet_lock in in_control and in6_control
with KERNEL_LOCK.
We also need to add some KERNEL_LOCKs to protect the network stack surely.
This is required, for example, for PR kern/51356.
Fix PR kern/53043
Initialized addresses of locks allocated by mutex_obj_alloc or rw_obj_alloc
were not useful because the addresses were mutex_obj_alloc or rw_obj_alloc
itself. What we want to know are callers of them.
(forgot to commit)
convert several raidframe ioctls to be bitsize idempotent so that
they work the same in 32 and 64 bit worlds, allowing netbsd32 to
configure and query raid properly. remove useless 'row' in a few
places. add COMPAT_80 and put the old ioctls there.
raidframeio.h:
RAIDFRAME_TEST_ACC
- remove, unused
RAIDFRAME_GET_COMPONENT_LABEL
- convert to label not pointer to label
RAIDFRAME_CHECK_RECON_STATUS_EXT
RAIDFRAME_CHECK_PARITYREWRITE_STATUS_EXT
RAIDFRAME_CHECK_COPYBACK_STATUS_EXT
- convert to progress info not pointer to info
RAIDFRAME_GET_INFO
- version entirely.
raidframevar.h:
- rf_recon_req{} has row, flags and raidPtr removed (they're
not a useful part of this interface.)
- RF_Config_s{} and RF_DeviceConfig_s{} have numRow/rows removed.
- RF_RaidDisk_s{} is re-ordered slightly to fix alignment
padding - the actual data was already OK.
- InstallSpareTable() loses row argument
rf_compat32.c has code for RF_Config_s{} in 32 bit mode, used
by RAIDFRAME_CONFIGURE and RAIDFRAME_GET_INFO32.
rf_compat80.c has code for rf_recon_req{}, RF_RaidDisk_s{} and
RF_DeviceConfig_s{} to handle RAIDFRAME_FAIL_DISK,
RAIDFRAME_GET_COMPONENT_LABEL, RAIDFRAME_CHECK_RECON_STATUS_EXT,
RAIDFRAME_CHECK_PARITYREWRITE_STATUS_EXT,
RAIDFRAME_CHECK_COPYBACK_STATUS_EXT, RAIDFRAME_GET_INFO.
move several of the per-ioctl code blocks into separate functions.
add rf_recon_req_internal{} to replace old usage of global
rf_recon_req{} that had unused void * in the structure, ruining
it's 32/64 bit ABI.
add missing case for RAIDFRAME_GET_INFO50.
adjust raid tests to use the new .conf format, and add a case to
test the old method as well.
raidctl:
deal with lack of 'row' members in a couple of places.
fail request no longer takes row.
handle "START array" sections with just "numCol numSpare", ie
no "numRow" specified. for now, generate old-style configuration
but update raidctl.8 to specify the new style (keeping reference
to the old style.)
note that: RF_ComponentLabel_s::{row,num_rows} and
RF_SingleComponent_s::row are obsolete but not removed yet.
On rump kernels, the callouts for domains, pffasttimo and pfslowtimo, started
before domains were attached. Normally the callouts were dispatched after
domain attaches (initializations) finished, however, under load the callouts
could be executed prior to the attaches, resulting in that the callouts accessed
unallocated or uninitialized resources.
This system call was used in legacy Lisp code, that was inherited to modern
age and still compiled against supported compat layers (e.g. in clisp,
oaklisp, Franz Lisp).
It used to instruct the kernel about paging policy (G/C aware, flush etc).
Newly compiled code (assuming that it will detect vadvise()) will use the
libc stub for vadvise(). The headers for this interface are gone.
vadvise(2) could be marked as COMPAT_80, but as long as we support ultrix,
sunos or aout68k ABI, don't bother with this.
Requested by <mrg>
Unlocking a not-owned mutex wasn't detected well (it could detect if the mutex
is not held by anyone but that's not enough). Let's check it (the check is the
same as normal kernel's mutex).
If LOCKDEBUG is enabled, give the check over LOCKDEBUG because it can provide
better debugging information.
Formerly rump kernels treated the two types of mutexes as both adaptive for
LOCKDEBUG for some reasons.
Now we can detect violations of mutex restrictions on rump kernels such as
taking an adaptive mutex with holding a spin mutex as well as normal kernels.
sstk - change stack section size
This functionality has never been implemented and is a remnant from 16-bit
UNIX. This stub appeared with the first NetBSD commit.
Sponsored by <The NetBSD Foundation>
The debugging feature detects violations of pserialize constraints.
It causes a panic:
- if a context switch happens in a read section, or
- if a sleepable function is called in a read section.
The feature is enabled only if LOCKDEBUG is on.
Discussed on tech-kern@
