NetBSD/sys/kern
ad 94843b1390 - Add and use wrapper functions that take and acquire page interlocks, and pairs
of page interlocks.  Require that the page interlock be held over calls to
  uvm_pageactivate(), uvm_pagewire() and similar.

- Solve the concurrency problem with page replacement state.  Rather than
  updating the global state synchronously, set an intended state on
  individual pages (active, inactive, enqueued, dequeued) while holding the
  page interlock.  After the interlock is released put the pages on a 128
  entry per-CPU queue for their state changes to be made real in batch.
  This results in in a ~400 fold decrease in contention on my test system.
  Proposed on tech-kern but modified to use the page interlock rather than
  atomics to synchronise as it's much easier to maintain that way, and
  cheaper.
2019-12-31 22:42:50 +00:00
..
bufq_disksort.c
bufq_fcfs.c
bufq_priocscan.c
bufq_readprio.c
cnmagic.c
compat_stub.c Move all non-emulation-specific coredump code into the coredump module, 2019-11-20 19:37:51 +00:00
core_elf32.c Introduce PT_LWPSTATUS + PT_LWPNEXT, obsolete PT_LWPINFO 2019-12-24 14:50:59 +00:00
core_elf64.c
core_netbsd.c Move all non-emulation-specific coredump code into the coredump module, 2019-11-20 19:37:51 +00:00
exec_aout.c Move all non-emulation-specific coredump code into the coredump module, 2019-11-20 19:37:51 +00:00
exec_ecoff.c Move all non-emulation-specific coredump code into the coredump module, 2019-11-20 19:37:51 +00:00
exec_elf32.c Move all non-emulation-specific coredump code into the coredump module, 2019-11-20 19:37:51 +00:00
exec_elf64.c Move all non-emulation-specific coredump code into the coredump module, 2019-11-20 19:37:51 +00:00
exec_elf.c PR/54549: ng0: always initialize execname. 2019-09-16 11:11:34 +00:00
exec_script.c adjust for new check_exec signature. 2019-09-15 20:21:12 +00:00
exec_subr.c
files.kern Put ptrace_read_lwpstatus() and process_read_lwpstatus() to a new file 2019-12-26 08:52:38 +00:00
genlintstub.awk
init_main.c Rename uvm_free() -> uvm_availmem(). 2019-12-31 13:07:09 +00:00
init_sysctl_base.c
init_sysctl.c remove kern.panic_now -- crashme panic node replaces it. 2019-01-15 07:11:23 +00:00
init_sysent.c regen 2019-10-21 14:23:53 +00:00
kern_acct.c fix flatly wrong indent 2019-05-26 19:23:04 +00:00
kern_auth.c Rename min/max -> uimin/uimax for better honesty. 2018-09-03 16:29:22 +00:00
kern_cctr.c Fix 'unknown' spellos 2019-06-24 06:24:33 +00:00
kern_cfglock.c
kern_clock.c Fix false sharing problems with cpu_info. Identified with tprof(8). 2019-12-01 15:34:44 +00:00
kern_condvar.c - Put back a microoptimisation that was accidentally removed. 2019-11-20 21:49:00 +00:00
kern_core.c Rather than keeping a separate mutex, condvar, and pserialize for each 2019-12-12 02:15:42 +00:00
kern_cpu.c Fix build failure. 2019-12-21 11:35:25 +00:00
kern_crashme.c clang does not like to deref a null pointer unless it is qualified volatile 2019-01-13 00:11:29 +00:00
kern_ctf.c merge a new version of the CDDL dtrace and ZFS code. 2018-05-28 21:04:59 +00:00
kern_descrip.c handle O_NOSIGPIPE too. 2019-02-20 19:42:14 +00:00
kern_drvctl.c - move export for devmon_insert_vec into sys/device.h. 2018-09-18 01:25:09 +00:00
kern_event.c print which process asked for an unsupported event so we can fix it. 2019-10-18 19:43:49 +00:00
kern_exec.c Make it possible to call mi_switch() and immediately switch to another CPU. 2019-12-06 21:36:10 +00:00
kern_exit.c Make it possible to call mi_switch() and immediately switch to another CPU. 2019-12-06 21:36:10 +00:00
kern_fileassoc.c
kern_fork.c - Extend the per-CPU counters matt@ did to include all of the hot counters 2019-12-16 22:47:54 +00:00
kern_history.c Traiing whitespace 2019-10-09 05:59:51 +00:00
kern_hook.c Add and use __FPTRCAST, requested by uwe@ 2019-10-16 18:29:49 +00:00
kern_idle.c - Add and use wrapper functions that take and acquire page interlocks, and pairs 2019-12-31 22:42:50 +00:00
kern_ksyms_buf.c
kern_ksyms.c use Elf_Sym ** instead of casting. 2017-11-04 22:17:55 +00:00
kern_kthread.c Minor scheduler cleanup: 2019-11-23 19:42:52 +00:00
kern_ktrace_vfs.c
kern_ktrace.c Rename min/max -> uimin/uimax for better honesty. 2018-09-03 16:29:22 +00:00
kern_lock.c Use __insn_barrier to enforce ordering in l_ncsw loops. 2019-12-03 15:20:59 +00:00
kern_lwp.c Make it possible to call mi_switch() and immediately switch to another CPU. 2019-12-06 21:36:10 +00:00
kern_malloc.c Add support for Kernel Memory Sanitizer (kMSan). It detects uninitialized 2019-11-14 16:23:52 +00:00
kern_module_hook.c Trailing whitespace 2019-12-13 08:02:53 +00:00
kern_module_vfs.c Merge the [pgoyette-compat] branch 2019-01-27 02:08:33 +00:00
kern_module.c Rename uvm_free() -> uvm_availmem(). 2019-12-31 13:07:09 +00:00
kern_mutex_obj.c Obtain proper initialized addresses of locks allocated by mutex_obj_alloc or rw_obj_alloc 2018-02-05 04:25:04 +00:00
kern_mutex.c Comment on previous explaining why it's needed. 2019-12-11 20:46:06 +00:00
kern_ntptime.c Zero out the ntptimeval structure to prevent a 4 byte kernel stack disclosure. 2018-10-29 22:02:25 +00:00
kern_pax.c
kern_physio.c Revert previous. No performance gain worth the potential headaches 2019-12-08 19:52:37 +00:00
kern_pmf.c expose pmf debug switches with sysctl. 2018-04-08 11:46:13 +00:00
kern_proc.c Rename uvm_free() -> uvm_availmem(). 2019-12-31 13:07:09 +00:00
kern_prot.c
kern_ras.c Remove charade dancing around now-dead RCU patent. 2019-12-14 16:58:25 +00:00
kern_rate.c
kern_reboot.c retire kern_xxx.c. long live kern_xxx.c. 2018-09-14 01:55:19 +00:00
kern_resource.c calcru: ignore running softints, unless softint_timing is on. 2019-11-21 17:50:49 +00:00
kern_rndpool.c - add or adjust /* FALLTHROUGH */ where appropriate 2019-02-03 03:19:25 +00:00
kern_rndq.c Do all delta calculations strictly using uint32_t. Avoid integer 2019-09-29 12:07:52 +00:00
kern_rndsink.c
kern_runq.c - Add some more failsafes to the CPU topology stuff, and build a 3rd 2019-12-03 22:28:41 +00:00
kern_rwlock_obj.c Obtain proper initialized addresses of locks allocated by mutex_obj_alloc or rw_obj_alloc 2018-02-05 04:25:04 +00:00
kern_rwlock.c Expunge the panicstr checks, we don't need them. 2019-12-09 21:02:10 +00:00
kern_scdebug.c syscall debug - fix build when SYSCALL_DEBUG option is present in kernel config file 2019-03-14 19:51:49 +00:00
kern_sdt.c
kern_sig.c Make it possible to call mi_switch() and immediately switch to another CPU. 2019-12-06 21:36:10 +00:00
kern_sleepq.c Fix LOCKDEBUG panic on mutex_init(). 2019-12-17 18:08:15 +00:00
kern_softint.c - Extend the per-CPU counters matt@ did to include all of the hot counters 2019-12-16 22:47:54 +00:00
kern_ssp.c
kern_stub.c steal an idea from uwe@ and implement gcc-8 function type cast 2019-10-08 07:33:14 +00:00
kern_subr.c Accept root device specification as NAME=label 2019-09-16 00:01:16 +00:00
kern_synch.c schedstate_percpu: add new flag SPCF_IDLE as a cheap and easy way to 2019-12-21 11:54:04 +00:00
kern_syscall.c xc_barrier - convenience function to xc_broadcast() a nop. 2019-10-06 15:11:16 +00:00
kern_sysctl.c Invert two conditions, to fix uninitialized memory access. If the node is 2019-07-03 17:31:32 +00:00
kern_tc.c s/inital/initial/ 2019-12-27 09:25:57 +00:00
kern_threadpool.c Use PRIu64 for "uint64_t tp_refcnt". 2019-01-17 10:18:52 +00:00
kern_time.c Check for valid timespec in clock_settime1() 2019-10-05 12:57:40 +00:00
kern_timeout.c Break the slow path for callout_halt() out into its own routine. 2019-11-21 17:57:40 +00:00
kern_todr.c The whole 'tv' structure gets added to the RND pool, so clear it first, 2019-07-07 15:12:59 +00:00
kern_turnstile.c Just make the turnstile locks statics and avoid the indirect reference 2019-12-16 19:22:15 +00:00
kern_uidinfo.c PR/53998: Joel Bertrand: Limit the number of semaphores on a 2019-03-01 03:03:19 +00:00
kern_uuid.c
kern_veriexec.c Add and use __FPTRCAST, requested by uwe@ 2019-10-16 18:29:49 +00:00
kgdb_stub.c
Make.tags.inc
Makefile
makesyscalls.sh do not try to rm sysautoload when it's not actually used for given compat; 2019-11-09 22:05:50 +00:00
sched_4bsd.c sched_tick(): don't try to optimise something that's called 10 times a 2019-12-06 18:33:19 +00:00
sched_m2.c sched_tick(): don't try to optimise something that's called 10 times a 2019-12-06 18:33:19 +00:00
subr_asan.c Add KASAN instrumentation on ucas and ufetch. 2019-10-10 13:45:14 +00:00
subr_autoconf.c Fix compile error. 2018-12-01 02:08:16 +00:00
subr_blist.c
subr_bufq.c rename module_name to strategy_module_name to avoid fatal shadowing of 2019-02-17 23:17:41 +00:00
subr_callback.c
subr_copy.c Exclude references to _ucas_{32,64}_mp() for _RUMPKERNEL. 2019-04-07 16:27:41 +00:00
subr_cprng.c Disable rngtest on output of cprng_strong. 2019-12-04 05:36:34 +00:00
subr_cpu.c Fix build break (ci->ci_dev is not available on every port). 2019-12-21 12:53:53 +00:00
subr_cpufreq.c
subr_csan.c Add KCSAN instrumentation for atomic_{load,store}_*. 2019-12-01 08:15:58 +00:00
subr_debug.c
subr_device.c
subr_devsw.c Add two utility functions to help use kmem with strings: kmem_strdupsize, 2017-11-07 18:35:57 +00:00
subr_disk_mbr.c Revert subr_disk_mbr.c r.1.54 2019-11-07 20:34:29 +00:00
subr_disk_open.c Fix vnode locking for opendisk(), must lock for VOP_OPEN(). 2019-02-20 10:02:51 +00:00
subr_disk.c kern/subr_disk: bounds_check_with_label: really protect against div by zero 2019-09-30 23:23:59 +00:00
subr_disklabel.c Fix previous. We define _KERNEL for rump in opt_rumpkernel.h. 2019-04-07 02:58:02 +00:00
subr_emul.c Merge the [pgoyette-compat] branch 2019-01-27 02:08:33 +00:00
subr_evcnt.c Fix kernel pointer leaks in sysctl_doevcnt. 2018-11-24 17:40:37 +00:00
subr_exec_fd.c Update comment to match existing function name. 2019-04-08 13:05:23 +00:00
subr_extent.c More diagnostic 2019-08-15 09:04:22 +00:00
subr_hash.c
subr_humanize.c This had a similar problem to that reported in PR lib/54053 2019-03-12 00:25:44 +00:00
subr_interrupt.c - don't return ENOMEM for errors not related to memory 2018-01-28 22:24:58 +00:00
subr_iostat.c Implement disk_rename()/iostat_rename() to rename a disk. 2019-05-22 08:47:02 +00:00
subr_ipi.c Add and use __FPTRCAST, requested by uwe@ 2019-10-16 18:29:49 +00:00
subr_kcov.c Revert the in_interrupt() change to use again the x86 specific code 2019-12-07 19:50:33 +00:00
subr_kcpuset.c Set kcpuset's bit correctly to avoid undefined behavior. Found by KUBSan. 2019-07-26 05:39:55 +00:00
subr_kleak.c Improve error handling, doesn't matter a lot, but still. 2018-12-10 07:24:49 +00:00
subr_kmem.c Add support for Kernel Memory Sanitizer (kMSan). It detects uninitialized 2019-11-14 16:23:52 +00:00
subr_kobj_vfs.c
subr_kobj.c add a kobj_error() to a recently added error case 2018-06-23 14:22:30 +00:00
subr_localcount.c Implement a debugging facility (overflow/underflow detection) for localcount 2017-11-17 09:26:36 +00:00
subr_lockdebug.c changes of r1.68 was reverted by r1.69. apply it again. 2019-05-28 07:39:16 +00:00
subr_log.c Rename min/max -> uimin/uimax for better honesty. 2018-09-03 16:29:22 +00:00
subr_lwp_specificdata.c Implement an aggressive psref leak detector 2019-05-17 03:34:26 +00:00
subr_msan.c Use the inlines; it is actually fine, since the compiler drops the inlines 2019-12-08 11:53:54 +00:00
subr_once.c add INIT_ONCE(9), FINI_ONCE(9) with changing once_t. 2019-03-19 08:16:51 +00:00
subr_optstr.c
subr_pcq.c pcq_create(): fix broken assertion. 2019-12-16 18:50:44 +00:00
subr_pcu.c
subr_percpu.c Allow equality in this assertion. 2019-12-05 03:21:08 +00:00
subr_physmap.c
subr_pool.c Switch to panic, and make the message more useful. 2019-12-27 15:49:20 +00:00
subr_prf.c put back line accidentally removed. 2019-05-21 04:57:02 +00:00
subr_prof.c Overhaul the API used to fetch and store individual memory cells in 2019-04-06 03:06:23 +00:00
subr_pserialize.c Switch psz_ev_excl to static evcnt. 2019-12-05 03:21:29 +00:00
subr_psref.c Implement an aggressive psref leak detector 2019-05-17 03:34:26 +00:00
subr_specificdata.c
subr_spldebug.c
subr_syscall_stats.c Merge the [pgoyette-compat] branch 2019-01-27 02:08:33 +00:00
subr_tftproot.c tftproot_getfile(): return E2BIG when bootfile is to long. 2019-01-20 21:26:13 +00:00
subr_thmap.c pass a pointer to atomic_cas_ptr_p(), not an (equiv) integer. 2019-02-04 08:00:27 +00:00
subr_time.c Avoid signed integer overflow in ts2timo() for ts->tv_nsec 2019-10-04 14:17:07 +00:00
subr_userconf.c call cnpollc(1) and cnpollc(0) around cngetc(). 2018-09-16 23:18:55 +00:00
subr_vmem.c Detangle the pagedaemon from uvm_fpageqlock: 2019-12-21 14:50:34 +00:00
subr_workqueue.c Don't wait on workqueue_wait if called from worker itself 2018-06-13 05:26:12 +00:00
subr_xcall.c Whitespace police (minor infraction). 2019-12-19 10:51:54 +00:00
sys_aio.c Minor scheduler cleanup: 2019-11-23 19:42:52 +00:00
sys_descrip.c Add F_GETPATH, presented to tech-kern. 2019-09-15 16:25:57 +00:00
sys_generic.c
sys_lwp.c Minor scheduler cleanup: 2019-11-23 19:42:52 +00:00
sys_module.c Rename the MODULE_*_HOOK() macros to MODULE_HOOK_*() as briefly 2019-03-01 11:06:55 +00:00
sys_mqueue.c Prevent O_EXEC for mq_open(2), and O_EXEC with a writable fd for open(2). 2019-09-15 20:51:03 +00:00
sys_pipe.c Handle half-closed pipes in FIONWRITE and FIONSPACE. 2019-04-26 17:24:23 +00:00
sys_process.c
sys_pset.c Simplify pset locking, making it easier to sync with LWP creation, etc. 2019-11-21 17:54:04 +00:00
sys_ptrace_common.c Put ptrace_read_lwpstatus() and process_read_lwpstatus() to a new file 2019-12-26 08:52:38 +00:00
sys_ptrace_lwpstatus.c Put ptrace_read_lwpstatus() and process_read_lwpstatus() to a new file 2019-12-26 08:52:38 +00:00
sys_ptrace.c Introduce PT_LWPSTATUS + PT_LWPNEXT, obsolete PT_LWPINFO 2019-12-24 14:50:59 +00:00
sys_sched.c
sys_select.c Minor correction to previous. 2019-11-22 20:04:03 +00:00
sys_sig.c Convert the sendsig_sigcontext_16 function pointer to use the new 2019-11-10 13:28:06 +00:00
sys_socket.c soo_fcntl is identical to fnullop_fcntl, use the latter 2018-12-04 00:18:05 +00:00
sys_syscall.c Introduce KLEAK, a new feature that can detect kernel information leaks. 2018-12-02 21:00:13 +00:00
syscalls_autoload.c regen 2019-09-22 23:03:20 +00:00
syscalls.c regen 2019-09-22 23:03:20 +00:00
syscalls.conf Add a new member to struct vfsstat and grow the unused members 2019-09-22 22:59:37 +00:00
syscalls.master Add a new member to struct vfsstat and grow the unused members 2019-09-22 22:59:37 +00:00
systrace_args.c regen 2019-09-22 23:03:20 +00:00
sysv_ipc.c Many years ago someone created a new __link_set_sysctl_funcs to hold 2019-08-07 00:38:01 +00:00
sysv_msg.c Avoid -LONG_MIN msgtyp in msgrcv(2) and treat it as LONG_MAX 2019-10-04 23:20:22 +00:00
sysv_sem.c Many years ago someone created a new __link_set_sysctl_funcs to hold 2019-08-07 00:38:01 +00:00
sysv_shm.c simpler fix for the race between shmat() and shmdt(): 2019-10-09 17:47:13 +00:00
tty_bsdpty.c
tty_conf.c
tty_ptm.c VOP_UNLOCK + vrele -> vput 2019-11-30 20:45:49 +00:00
tty_pty.c Fix reporting EOF via kevent and add a test case 2019-02-15 18:57:15 +00:00
tty_subr.c s/transfered/transferred/ 2019-12-27 09:41:48 +00:00
tty_tty.c
tty.c Rename the MODULE_*_HOOK() macros to MODULE_HOOK_*() as briefly 2019-03-01 11:06:55 +00:00
uipc_accf.c
uipc_domain.c Fix apparent race. 2018-12-27 07:56:43 +00:00
uipc_mbuf.c Minor changes, reported by the LGTM bot. 2019-12-06 07:27:06 +00:00
uipc_mbufdebug.c KNF. No functional change. 2018-10-18 05:44:19 +00:00
uipc_proto.c
uipc_sem.c ksem_sysinit(): more lossage. 2019-12-17 18:16:05 +00:00
uipc_socket2.c Fix info leaks: the alignment of the structures causes uninitialized heap 2019-07-11 17:30:44 +00:00
uipc_socket.c Add a check before the memcpy. memcpy is defined to never take NULL as 2019-10-14 16:27:03 +00:00
uipc_syscalls.c sys_recvmmsg: don't defer an error that already gets returned. 2018-11-12 09:21:13 +00:00
uipc_usrreq.c Fix info leak: the padding after the header causes uninitialized heap 2019-07-29 09:42:17 +00:00
vfs_bio.c Rename uvm_free() -> uvm_availmem(). 2019-12-31 13:07:09 +00:00
vfs_cache.c Back out previous temporarily - seeing unusual lookup failures. Will 2019-12-01 18:31:19 +00:00
vfs_cwd.c
vfs_dirhash.c Introduce PR_ZERO to avoid open-coding memset()s everywhere. OK riastradh@. 2019-02-10 17:13:33 +00:00
vfs_getcwd.c - add missing error check 2019-09-14 21:23:34 +00:00
vfs_hooks.c
vfs_init.c Move pnbuf_cache into vfs_init.c, where it belongs. 2019-03-28 18:12:24 +00:00
vfs_lockf.c
vfs_lookup.c Make namei() work with no root dir yet. 2019-07-18 09:39:40 +00:00
vfs_mount.c Make mntvnode_lock per-mount, and address false sharing of struct mount. 2019-12-22 19:47:34 +00:00
vfs_quotactl.c
vfs_subr.c Make mntvnode_lock per-mount, and address false sharing of struct mount. 2019-12-22 19:47:34 +00:00
vfs_syscalls.c sys_fchdir: use LK_SHARED. 2019-12-31 11:49:08 +00:00
vfs_trans.c Add an owner field to fstrans mount info and use it to hold 2019-06-17 08:07:27 +00:00
vfs_vnode.c - Extend the per-CPU counters matt@ did to include all of the hot counters 2019-12-16 22:47:54 +00:00
vfs_vnops.c - Extend the per-CPU counters matt@ did to include all of the hot counters 2019-12-16 22:47:54 +00:00
vfs_wapbl.c constify wapbl_ops 2018-12-10 21:19:33 +00:00
vfs_xattr.c
vnode_if.c Regen for VOP_LOCK & LK_UPGRADE/LK_DOWNGRADE. 2019-12-01 13:58:52 +00:00
vnode_if.sh Minor vnode locking changes: 2019-12-01 13:56:29 +00:00
vnode_if.src As VOP_STRATEGY() usually calls itself on the file system holding "/dev" 2019-10-11 08:04:52 +00:00