Aren
/
NetBSD
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
NetBSD/sys/kern/kern_event.c

3040 lines
74 KiB
C
Raw Normal View History

s/ for for / for / in comment.
2023-09-21 12:31:49 +03:00
/* $NetBSD: kern_event.c,v 1.150 2023/09/21 09:31:50 msaitoh Exp $ */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
/*-
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
* Copyright (c) 2008, 2009, 2021 The NetBSD Foundation, Inc.
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* All rights reserved.
*
Add fileops::fo_drain(), to be called from fd_close() when there is more than one active reference to a file descriptor. It should dislodge threads sleeping while holding a reference to the descriptor. Implemented only for sockets but should be extended to pipes, fifos, etc. Fixes the case of a multithreaded process doing something like the following, which would have hung until the process got a signal. thr0 accept(fd, ...) thr1 close(fd)
2009-04-04 14:12:51 +04:00
* This code is derived from software contributed to The NetBSD Foundation
* by Andrew Doran.
*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
merge FreeBSD timecounters from branch simonb-timecounters - struct timeval time is gone time.tv_sec -> time_second - struct timeval mono_time is gone mono_time.tv_sec -> time_uptime - access to time via {get,}{micro,nano,bin}time() get* versions are fast but less precise - support NTP nanokernel implementation (NTP API 4) - further reading: Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
2006-06-08 02:33:33 +04:00
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*-
* Copyright (c) 1999,2000,2001 Jonathan Lemon <jlemon@FreeBSD.org>
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
* Copyright (c) 2009 Apple, Inc
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* FreeBSD: src/sys/kern/kern_event.c,v 1.27 2001/07/05 17:10:44 rwatson Exp
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
Check _KERNEL_OPT before including opt_ddb.h.
2021-10-10 22:11:56 +03:00
#ifdef _KERNEL_OPT
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
#include "opt_ddb.h"
Check _KERNEL_OPT before including opt_ddb.h.
2021-10-10 22:11:56 +03:00
#endif /* _KERNEL_OPT */
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
add __KERNEL_RCSID()
2003-06-23 17:14:49 +04:00
#include <sys/cdefs.h>
s/ for for / for / in comment.
2023-09-21 12:31:49 +03:00
__KERNEL_RCSID(0, "$NetBSD: kern_event.c,v 1.150 2023/09/21 09:31:50 msaitoh Exp $");
add __KERNEL_RCSID()
2003-06-23 17:14:49 +04:00
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
#include <sys/param.h>
#include <sys/systm.h>
#include <sys/kernel.h>
Split p_xstat (composite wait(2) status code, or signal number depending on context) into: 1. p_xexit: exit code 2. p_xsig: signal number 3. p_sflag & WCOREFLAG bit to indicated that the process core-dumped. Fix the documentation of the flag bits in <sys/proc.h>
2016-04-04 23:47:57 +03:00
#include <sys/wait.h>
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
#include <sys/proc.h>
#include <sys/file.h>
#include <sys/select.h>
#include <sys/queue.h>
#include <sys/event.h>
#include <sys/eventvar.h>
#include <sys/poll.h>
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
#include <sys/kmem.h>
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
#include <sys/stat.h>
#include <sys/filedesc.h>
#include <sys/syscallargs.h>
integrate kauth.
2006-05-15 01:15:11 +04:00
#include <sys/kauth.h>
+#include <sys/conf.h>
2007-07-21 23:23:03 +04:00
#include <sys/conf.h>
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
#include <sys/atomic.h>
static int kqueue_scan(file_t *, size_t, struct kevent *,
const struct timespec *, register_t *,
const struct kevent_ops *, struct kevent *,
size_t);
static int kqueue_ioctl(file_t *, u_long, void *);
static int kqueue_fcntl(file_t *, u_int, void *);
static int kqueue_poll(file_t *, int);
static int kqueue_kqfilter(file_t *, struct knote *);
static int kqueue_stat(file_t *, struct stat *);
static int kqueue_close(file_t *);
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
static void kqueue_restart(file_t *);
file(9): New fo_fpathconf operation. XXX kernel revbump -- struct fileops API and ABI change
2023-04-22 16:52:54 +03:00
static int kqueue_fpathconf(file_t *, int, register_t *);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static int kqueue_register(struct kqueue *, struct kevent *);
static void kqueue_doclose(struct kqueue *, struct klist *, int);
static void knote_detach(struct knote *, filedesc_t *fdp, bool);
static void knote_enqueue(struct knote *);
static void knote_activate(struct knote *);
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
static void knote_activate_locked(struct knote *);
Support modifying an existing timer without having to delete it first. Semantics match FreeBSD.
2021-10-22 07:49:24 +03:00
static void knote_deactivate_locked(struct knote *);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static void filt_kqdetach(struct knote *);
static int filt_kqueue(struct knote *, long hint);
static int filt_procattach(struct knote *);
static void filt_procdetach(struct knote *);
static int filt_proc(struct knote *, long hint);
static int filt_fileattach(struct knote *);
static void filt_timerexpire(void *x);
static int filt_timerattach(struct knote *);
static void filt_timerdetach(struct knote *);
static int filt_timer(struct knote *, long hint);
Support modifying an existing timer without having to delete it first. Semantics match FreeBSD.
2021-10-22 07:49:24 +03:00
static int filt_timertouch(struct knote *, struct kevent *, long type);
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
static int filt_userattach(struct knote *);
static void filt_userdetach(struct knote *);
static int filt_user(struct knote *, long hint);
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
static int filt_usertouch(struct knote *, struct kevent *, long type);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
/*
* Private knote state that should never be exposed outside
* of kern_event.c
*
* Field locking:
*
* q kn_kq->kq_lock
*/
struct knote_impl {
struct knote ki_knote;
unsigned int ki_influx; /* q: in-flux counter */
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
kmutex_t ki_foplock; /* for kn_filterops */
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
};
#define KIMPL_TO_KNOTE(kip) (&(kip)->ki_knote)
#define KNOTE_TO_KIMPL(knp) container_of((knp), struct knote_impl, ki_knote)
static inline struct knote *
knote_alloc(bool sleepok)
{
struct knote_impl *ki;
ki = kmem_zalloc(sizeof(*ki), sleepok ? KM_SLEEP : KM_NOSLEEP);
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
mutex_init(&ki->ki_foplock, MUTEX_DEFAULT, IPL_NONE);
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
return KIMPL_TO_KNOTE(ki);
}
static inline void
knote_free(struct knote *kn)
{
struct knote_impl *ki = KNOTE_TO_KIMPL(kn);
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
mutex_destroy(&ki->ki_foplock);
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
kmem_free(ki, sizeof(*ki));
}
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
static inline void
knote_foplock_enter(struct knote *kn)
{
mutex_enter(&KNOTE_TO_KIMPL(kn)->ki_foplock);
}
static inline void
knote_foplock_exit(struct knote *kn)
{
mutex_exit(&KNOTE_TO_KIMPL(kn)->ki_foplock);
}
kern_event.c: Mark KASSERT-only static function as __diagused. Otherwise clang objects with -Wunneeded-internal-declaration.
2022-07-24 22:23:44 +03:00
static inline bool __diagused
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
knote_foplock_owned(struct knote *kn)
{
return mutex_owned(&KNOTE_TO_KIMPL(kn)->ki_foplock);
}
Cloning cleanup: 1. make fileops const 2. add 2 new negative errno's to `officially' support the cloning hack: - EDUPFD (used to overload ENODEV) - EMOVEFD (used to overload ENXIO) 3. Created an fdclone() function to encapsulate the operations needed for EMOVEFD, and made all cloners use it. 4. Centralize the local noop/badop fileops functions to: fnullop_fcntl, fnullop_poll, fnullop_kqfilter, fbadop_stat
2004-11-30 07:25:43 +03:00
static const struct fileops kqueueops = {
add fo_name so we can identify the fileops in a simple way.
2017-11-30 23:25:54 +03:00
.fo_name = "kqueue",
Add fileops::fo_drain(), to be called from fd_close() when there is more than one active reference to a file descriptor. It should dislodge threads sleeping while holding a reference to the descriptor. Implemented only for sockets but should be extended to pipes, fifos, etc. Fixes the case of a multithreaded process doing something like the following, which would have hung until the process got a signal. thr0 accept(fd, ...) thr1 close(fd)
2009-04-04 14:12:51 +04:00
.fo_read = (void *)enxio,
.fo_write = (void *)enxio,
.fo_ioctl = kqueue_ioctl,
.fo_fcntl = kqueue_fcntl,
.fo_poll = kqueue_poll,
.fo_stat = kqueue_stat,
.fo_close = kqueue_close,
.fo_kqfilter = kqueue_kqfilter,
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
.fo_restart = kqueue_restart,
file(9): New fo_fpathconf operation. XXX kernel revbump -- struct fileops API and ABI change
2023-04-22 16:52:54 +03:00
.fo_fpathconf = kqueue_fpathconf,
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
};
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
static void
filt_nopdetach(struct knote *kn __unused)
{
}
static int
filt_nopevent(struct knote *kn __unused, long hint __unused)
{
return 0;
}
static const struct filterops nop_fd_filtops = {
.f_flags = FILTEROP_ISFD | FILTEROP_MPSAFE,
.f_attach = NULL,
.f_detach = filt_nopdetach,
.f_event = filt_nopevent,
};
static const struct filterops nop_filtops = {
.f_flags = FILTEROP_MPSAFE,
.f_attach = NULL,
.f_detach = filt_nopdetach,
.f_event = filt_nopevent,
};
Use C99 initializer for filterops Mostly done with spatch with touchups for indentation @@ expression a; identifier b,c,d; identifier p; @@ const struct filterops p = - { a, b, c, d + { + .f_isfd = a, + .f_attach = b, + .f_detach = c, + .f_event = d, };
2017-10-25 11:12:37 +03:00
static const struct filterops kqread_filtops = {
Mark kqread_filtops, user_filtops, and seltrue_filtops as MPSAFE.
2021-09-26 21:13:58 +03:00
.f_flags = FILTEROP_ISFD | FILTEROP_MPSAFE,
Use C99 initializer for filterops Mostly done with spatch with touchups for indentation @@ expression a; identifier b,c,d; identifier p; @@ const struct filterops p = - { a, b, c, d + { + .f_isfd = a, + .f_attach = b, + .f_detach = c, + .f_event = d, };
2017-10-25 11:12:37 +03:00
.f_attach = NULL,
.f_detach = filt_kqdetach,
.f_event = filt_kqueue,
};
static const struct filterops proc_filtops = {
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
.f_flags = FILTEROP_MPSAFE,
Use C99 initializer for filterops Mostly done with spatch with touchups for indentation @@ expression a; identifier b,c,d; identifier p; @@ const struct filterops p = - { a, b, c, d + { + .f_isfd = a, + .f_attach = b, + .f_detach = c, + .f_event = d, };
2017-10-25 11:12:37 +03:00
.f_attach = filt_procattach,
.f_detach = filt_procdetach,
.f_event = filt_proc,
};
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
/*
* file_filtops is not marked MPSAFE because it's going to call
* fileops::fo_kqfilter(), which might not be. That function,
* however, will override the knote's filterops, and thus will
* inherit the MPSAFE-ness of the back-end at that time.
*/
Use C99 initializer for filterops Mostly done with spatch with touchups for indentation @@ expression a; identifier b,c,d; identifier p; @@ const struct filterops p = - { a, b, c, d + { + .f_isfd = a, + .f_attach = b, + .f_detach = c, + .f_event = d, };
2017-10-25 11:12:37 +03:00
static const struct filterops file_filtops = {
Change the kqueue filterops::f_isfd field to filterops::f_flags, and define a flag FILTEROP_ISFD that has the meaning of the prior f_isfd. Field and flag name aligned with OpenBSD. This does not constitute a functional or ABI change, as the field location and size, and the value placed in that field, are the same as the previous code, but we're bumping __NetBSD_Version__ so 3rd-party module source code can adapt, as needed. NetBSD 9.99.89
2021-09-26 04:16:07 +03:00
.f_flags = FILTEROP_ISFD,
Use C99 initializer for filterops Mostly done with spatch with touchups for indentation @@ expression a; identifier b,c,d; identifier p; @@ const struct filterops p = - { a, b, c, d + { + .f_isfd = a, + .f_attach = b, + .f_detach = c, + .f_event = d, };
2017-10-25 11:12:37 +03:00
.f_attach = filt_fileattach,
.f_detach = NULL,
.f_event = NULL,
};
static const struct filterops timer_filtops = {
- Rename kqueue_misc_lock -> kqueue_timer_lock, since EVFILT_TIMER is now its only user. Also initialize it as IPL_SOFTCLOCK; there is no practical difference in how it operates (it is still an adaptive lock), but this serves as a visual reminder that we are interlocking against a callout. - Add some comments that describe why we don't need to hold kqueue_timer_lock when detaching an EVFILT_TIMER due to guarantees made by callout_halt(). - Mark timer_filtops as MPSAFE.
2021-09-27 02:34:46 +03:00
.f_flags = FILTEROP_MPSAFE,
Use C99 initializer for filterops Mostly done with spatch with touchups for indentation @@ expression a; identifier b,c,d; identifier p; @@ const struct filterops p = - { a, b, c, d + { + .f_isfd = a, + .f_attach = b, + .f_detach = c, + .f_event = d, };
2017-10-25 11:12:37 +03:00
.f_attach = filt_timerattach,
.f_detach = filt_timerdetach,
.f_event = filt_timer,
Support modifying an existing timer without having to delete it first. Semantics match FreeBSD.
2021-10-22 07:49:24 +03:00
.f_touch = filt_timertouch,
Use C99 initializer for filterops Mostly done with spatch with touchups for indentation @@ expression a; identifier b,c,d; identifier p; @@ const struct filterops p = - { a, b, c, d + { + .f_isfd = a, + .f_attach = b, + .f_detach = c, + .f_event = d, };
2017-10-25 11:12:37 +03:00
};
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
static const struct filterops user_filtops = {
Mark kqread_filtops, user_filtops, and seltrue_filtops as MPSAFE.
2021-09-26 21:13:58 +03:00
.f_flags = FILTEROP_MPSAFE,
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
.f_attach = filt_userattach,
.f_detach = filt_userdetach,
.f_event = filt_user,
.f_touch = filt_usertouch,
};
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static u_int kq_ncallouts = 0;
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
static int kq_calloutmax = (4 * 1024);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
#define KN_HASHSIZE 64 /* XXX should be tunable */
#define KN_HASH(val, mask) (((val) ^ (val >> 8)) & (mask))
Fix the locking around EVFILT_FS. Previously, the code would walk the fs_klist and take the kqueue_misc_lock inside the event callback. However, that list can be modified by the attach and detach callbacks, which could result in the walker stepping right off a cliff. Instead, we give the fs_klist it's own lock, and hold it while we call knote(), using the NOTE_SUBMIT protocol. Also, fs_filtops into vfs_syscalls.c so all of the locking logic is contained in one file (there is precedence with sig_filtops). fs_filtops is now marked MPSAFE.
2021-09-27 00:29:38 +03:00
extern const struct filterops fs_filtops; /* vfs_syscalls.c */
extern const struct filterops sig_filtops; /* kern_sig.c */
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
s/ for for / for / in comment.
2023-09-21 12:31:49 +03:00
* Table for all system-defined filters.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
* These should be listed in the numeric order of the EVFILT_* defines.
* If filtops is NULL, the filter isn't implemented in NetBSD.
* End of list is when name is NULL.
Nix trailing whitespace. No functional change.
2017-07-03 03:53:33 +03:00
*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* Note that 'refcnt' is meaningless for built-in filters.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
struct kfilter {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
const char *name; /* name of filter */
uint32_t filter; /* id of filter */
unsigned refcnt; /* reference count */
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
const struct filterops *filtops;/* operations for filter */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
size_t namelen; /* length of name string */
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
};
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
/* System defined filters */
static struct kfilter sys_kfilters[] = {
{ "EVFILT_READ", EVFILT_READ, 0, &file_filtops, 0 },
{ "EVFILT_WRITE", EVFILT_WRITE, 0, &file_filtops, 0, },
{ "EVFILT_AIO", EVFILT_AIO, 0, NULL, 0 },
{ "EVFILT_VNODE", EVFILT_VNODE, 0, &file_filtops, 0 },
{ "EVFILT_PROC", EVFILT_PROC, 0, &proc_filtops, 0 },
{ "EVFILT_SIGNAL", EVFILT_SIGNAL, 0, &sig_filtops, 0 },
{ "EVFILT_TIMER", EVFILT_TIMER, 0, &timer_filtops, 0 },
Merge autofs support from: Tomohiro Kusumi XXX: Does not work yet
2018-01-09 06:31:12 +03:00
{ "EVFILT_FS", EVFILT_FS, 0, &fs_filtops, 0 },
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
{ "EVFILT_USER", EVFILT_USER, 0, &user_filtops, 0 },
Add support for the EVFILT_EMPTY filter, which is activated when the write buffer associated with the file descriptor is empty. This is currently implemented only for sockets, and is intended primarily to provide visibility to applications that all previously written data has been acknowledged by the TCP layer on the receiver. Compatible with the same filter in FreeBSD.
2021-10-23 04:28:33 +03:00
{ "EVFILT_EMPTY", EVFILT_EMPTY, 0, &file_filtops, 0 },
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
{ NULL, 0, 0, NULL, 0 },
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
};
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
/* User defined kfilters */
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
static struct kfilter *user_kfilters; /* array */
static int user_kfilterc; /* current offset */
static int user_kfiltermaxc; /* max size so far */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static size_t user_kfiltersz; /* size of allocated memory */
Document lock order and locking rules.
2017-10-25 09:02:40 +03:00
/*
* Global Locks.
*
* Lock order:
*
* kqueue_filter_lock
* -> kn_kq->kq_fdp->fd_lock
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
* -> knote foplock (if taken)
- Rename kqueue_misc_lock -> kqueue_timer_lock, since EVFILT_TIMER is now its only user. Also initialize it as IPL_SOFTCLOCK; there is no practical difference in how it operates (it is still an adaptive lock), but this serves as a visual reminder that we are interlocking against a callout. - Add some comments that describe why we don't need to hold kqueue_timer_lock when detaching an EVFILT_TIMER due to guarantees made by callout_halt(). - Mark timer_filtops as MPSAFE.
2021-09-27 02:34:46 +03:00
* -> object lock (e.g., device driver lock, &c.)
Document lock order and locking rules.
2017-10-25 09:02:40 +03:00
* -> kn_kq->kq_lock
*
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
* Locking rules. ==> indicates the lock is acquired by the backing
* object, locks prior are acquired before calling filter ops:
*
* f_attach: fdp->fd_lock -> knote foplock ->
* (maybe) KERNEL_LOCK ==> backing object lock
Document lock order and locking rules.
2017-10-25 09:02:40 +03:00
*
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
* f_detach: fdp->fd_lock -> knote foplock ->
* (maybe) KERNEL_LOCK ==> backing object lock
*
* f_event via kevent: fdp->fd_lock -> knote foplock ->
* (maybe) KERNEL_LOCK ==> backing object lock
* N.B. NOTE_SUBMIT will never be set in the "hint" argument
* in this case.
*
* f_event via knote (via backing object: Whatever caller guarantees.
* Typically:
* f_event(NOTE_SUBMIT): caller has already acquired backing
* object lock.
* f_event(!NOTE_SUBMIT): caller has not acquired backing object,
* lock or has possibly acquired KERNEL_LOCK. Backing object
* lock may or may not be acquired as-needed.
* N.B. the knote foplock will **not** be acquired in this case. The
* caller guarantees that klist_fini() will not be called concurrently
* with knote().
*
* f_touch: fdp->fd_lock -> kn_kq->kq_lock (spin lock)
* N.B. knote foplock is **not** acquired in this case and
* the caller must guarantee that klist_fini() will never
* be called. kevent_register() restricts filters that
* provide f_touch to known-safe cases.
*
* klist_fini(): Caller must guarantee that no more knotes can
* be attached to the klist, and must **not** hold the backing
* object's lock; klist_fini() itself will acquire the foplock
* of each knote on the klist.
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
*
* Locking rules when detaching knotes:
*
* There are some situations where knote submission may require dropping
* locks (see knote_proc_fork()). In order to support this, it's possible
* to mark a knote as being 'in-flux'. Such a knote is guaranteed not to
* be detached while it remains in-flux. Because it will not be detached,
* locks can be dropped so e.g. memory can be allocated, locks on other
* data structures can be acquired, etc. During this time, any attempt to
* detach an in-flux knote must wait until the knote is no longer in-flux.
* When this happens, the knote is marked for death (KN_WILLDETACH) and the
* LWP who gets to finish the detach operation is recorded in the knote's
* 'udata' field (which is no longer required for its original purpose once
* a knote is so marked). Code paths that lead to knote_detach() must ensure
* that their LWP is the one tasked with its final demise after waiting for
* the in-flux status of the knote to clear. Note that once a knote is
* marked KN_WILLDETACH, no code paths may put it into an in-flux state.
*
* Once the special circumstances have been handled, the locks are re-
* acquired in the proper order (object lock -> kq_lock), the knote taken
* out of flux, and any waiters are notified. Because waiters must have
* also dropped *their* locks in order to safely block, they must re-
* validate all of their assumptions; see knote_detach_quiesce(). See also
* the kqueue_register() (EV_ADD, EV_DELETE) and kqueue_scan() (EV_ONESHOT)
* cases.
*
* When kqueue_scan() encounters an in-flux knote, the situation is
* treated like another LWP's list marker.
*
* LISTEN WELL: It is important to not hold knotes in flux for an
* extended period of time! In-flux knotes effectively block any
* progress of the kqueue_scan() operation. Any code paths that place
* knotes in-flux should be careful to not block for indefinite periods
* of time, such as for memory allocation (i.e. KM_NOSLEEP is OK, but
* KM_SLEEP is not).
Document lock order and locking rules.
2017-10-25 09:02:40 +03:00
*/
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static krwlock_t kqueue_filter_lock; /* lock on filter lists */
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
#define KQ_FLUX_WAIT(kq) (void)cv_wait(&kq->kq_cv, &kq->kq_lock)
#define KQ_FLUX_WAKEUP(kq) cv_broadcast(&kq->kq_cv)
static inline bool
kn_in_flux(struct knote *kn)
{
KASSERT(mutex_owned(&kn->kn_kq->kq_lock));
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
return KNOTE_TO_KIMPL(kn)->ki_influx != 0;
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
static inline bool
kn_enter_flux(struct knote *kn)
{
KASSERT(mutex_owned(&kn->kn_kq->kq_lock));
if (kn->kn_status & KN_WILLDETACH) {
return false;
}
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
struct knote_impl *ki = KNOTE_TO_KIMPL(kn);
KASSERT(ki->ki_influx < UINT_MAX);
ki->ki_influx++;
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
return true;
}
static inline bool
kn_leave_flux(struct knote *kn)
{
KASSERT(mutex_owned(&kn->kn_kq->kq_lock));
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
struct knote_impl *ki = KNOTE_TO_KIMPL(kn);
KASSERT(ki->ki_influx > 0);
ki->ki_influx--;
return ki->ki_influx == 0;
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
static void
kn_wait_flux(struct knote *kn, bool can_loop)
{
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
struct knote_impl *ki = KNOTE_TO_KIMPL(kn);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
bool loop;
KASSERT(mutex_owned(&kn->kn_kq->kq_lock));
/*
* It may not be safe for us to touch the knote again after
* dropping the kq_lock. The caller has let us know in
* 'can_loop'.
*/
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
for (loop = true; loop && ki->ki_influx != 0; loop = can_loop) {
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
KQ_FLUX_WAIT(kn->kn_kq);
}
}
#define KNOTE_WILLDETACH(kn) \
do { \
(kn)->kn_status |= KN_WILLDETACH; \
(kn)->kn_kevent.udata = curlwp; \
} while (/*CONSTCOND*/0)
/*
* Wait until the specified knote is in a quiescent state and
* safe to detach. Returns true if we potentially blocked (and
* thus dropped our locks).
*/
static bool
knote_detach_quiesce(struct knote *kn)
{
struct kqueue *kq = kn->kn_kq;
filedesc_t *fdp = kq->kq_fdp;
KASSERT(mutex_owned(&fdp->fd_lock));
mutex_spin_enter(&kq->kq_lock);
/*
* There are two cases where we might see KN_WILLDETACH here:
*
* 1. Someone else has already started detaching the knote but
* had to wait for it to settle first.
*
* 2. We had to wait for it to settle, and had to come back
* around after re-acquiring the locks.
*
* When KN_WILLDETACH is set, we also set the LWP that claimed
* the prize of finishing the detach in the 'udata' field of the
* knote (which will never be used again for its usual purpose
* once the note is in this state). If it doesn't point to us,
* we must drop the locks and let them in to finish the job.
*
* Otherwise, once we have claimed the knote for ourselves, we
* can finish waiting for it to settle. The is the only scenario
* where touching a detaching knote is safe after dropping the
* locks.
*/
if ((kn->kn_status & KN_WILLDETACH) != 0 &&
kn->kn_kevent.udata != curlwp) {
/*
* N.B. it is NOT safe for us to touch the knote again
* after dropping the locks here. The caller must go
* back around and re-validate everything. However, if
* the knote is in-flux, we want to block to minimize
* busy-looping.
*/
mutex_exit(&fdp->fd_lock);
if (kn_in_flux(kn)) {
kn_wait_flux(kn, false);
mutex_spin_exit(&kq->kq_lock);
return true;
}
mutex_spin_exit(&kq->kq_lock);
preempt_point();
return true;
}
/*
* If we get here, we know that we will be claiming the
* detach responsibilies, or that we already have and
* this is the second attempt after re-validation.
*/
KASSERT((kn->kn_status & KN_WILLDETACH) == 0 ||
kn->kn_kevent.udata == curlwp);
/*
* Similarly, if we get here, either we are just claiming it
* and may have to wait for it to settle, or if this is the
* second attempt after re-validation that no other code paths
* have put it in-flux.
*/
KASSERT((kn->kn_status & KN_WILLDETACH) == 0 ||
kn_in_flux(kn) == false);
KNOTE_WILLDETACH(kn);
if (kn_in_flux(kn)) {
mutex_exit(&fdp->fd_lock);
kn_wait_flux(kn, true);
/*
* It is safe for us to touch the knote again after
* dropping the locks, but the caller must still
* re-validate everything because other aspects of
* the environment may have changed while we blocked.
*/
KASSERT(kn_in_flux(kn) == false);
mutex_spin_exit(&kq->kq_lock);
return true;
}
mutex_spin_exit(&kq->kq_lock);
return false;
}
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
/*
* Calls into the filterops need to be resilient against things which
* destroy a klist, e.g. device detach, freeing a vnode, etc., to avoid
* chasing garbage pointers (to data, or even potentially code in a
* module about to be unloaded). To that end, we acquire the
* knote foplock before calling into the filter ops. When a driver
* (or anything else) is tearing down its klist, klist_fini() enumerates
* each knote, acquires its foplock, and replaces the filterops with a
* nop stub, allowing knote detach (when descriptors are closed) to safely
* proceed.
*/
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
static int
filter_attach(struct knote *kn)
{
int rv;
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
KASSERT(knote_foplock_owned(kn));
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
KASSERT(kn->kn_fop != NULL);
KASSERT(kn->kn_fop->f_attach != NULL);
/*
* N.B. that kn->kn_fop may change as the result of calling
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
* f_attach(). After f_attach() returns, kn->kn_fop may not
* be modified by code outside of klist_fini().
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
*/
if (kn->kn_fop->f_flags & FILTEROP_MPSAFE) {
rv = kn->kn_fop->f_attach(kn);
} else {
KERNEL_LOCK(1, NULL);
rv = kn->kn_fop->f_attach(kn);
KERNEL_UNLOCK_ONE(NULL);
}
return rv;
}
static void
filter_detach(struct knote *kn)
{
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
KASSERT(knote_foplock_owned(kn));
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
KASSERT(kn->kn_fop != NULL);
KASSERT(kn->kn_fop->f_detach != NULL);
if (kn->kn_fop->f_flags & FILTEROP_MPSAFE) {
kn->kn_fop->f_detach(kn);
} else {
KERNEL_LOCK(1, NULL);
kn->kn_fop->f_detach(kn);
KERNEL_UNLOCK_ONE(NULL);
}
}
static int
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
filter_event(struct knote *kn, long hint, bool submitting)
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
{
int rv;
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
/* See knote(). */
KASSERT(submitting || knote_foplock_owned(kn));
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
KASSERT(kn->kn_fop != NULL);
KASSERT(kn->kn_fop->f_event != NULL);
if (kn->kn_fop->f_flags & FILTEROP_MPSAFE) {
rv = kn->kn_fop->f_event(kn, hint);
} else {
KERNEL_LOCK(1, NULL);
rv = kn->kn_fop->f_event(kn, hint);
KERNEL_UNLOCK_ONE(NULL);
}
return rv;
}
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
static int
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
filter_touch(struct knote *kn, struct kevent *kev, long type)
{
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
/*
* XXX We cannot assert that the knote foplock is held here
* XXX beause we cannot safely acquire it in all cases
* XXX where "touch" will be used in kqueue_scan(). We just
* XXX have to assume that f_touch will always be safe to call,
* XXX and kqueue_register() allows only the two known-safe
* XXX users of that op.
*/
KASSERT(kn->kn_fop != NULL);
KASSERT(kn->kn_fop->f_touch != NULL);
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
return kn->kn_fop->f_touch(kn, kev, type);
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
}
Move kevent policy back to the subsystem.
2009-10-03 04:14:07 +04:00
static kauth_listener_t kqueue_listener;
static int
kqueue_listener_cb(kauth_cred_t cred, kauth_action_t action, void *cookie,
void *arg0, void *arg1, void *arg2, void *arg3)
{
struct proc *p;
int result;
result = KAUTH_RESULT_DEFER;
p = arg0;
if (action != KAUTH_PROCESS_KEVENT_FILTER)
return result;
if ((kauth_cred_getuid(p->p_cred) != kauth_cred_getuid(cred) ||
ISSET(p->p_flag, PK_SUGID)))
return result;
result = KAUTH_RESULT_ALLOW;
return result;
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
/*
* Initialize the kqueue subsystem.
*/
void
kqueue_init(void)
{
rw_init(&kqueue_filter_lock);
Move kevent policy back to the subsystem.
2009-10-03 04:14:07 +04:00
kqueue_listener = kauth_listen_scope(KAUTH_SCOPE_PROCESS,
kqueue_listener_cb, NULL);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
* Find kfilter entry by name, or NULL if not found.
*/
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static struct kfilter *
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kfilter_byname_sys(const char *name)
{
int i;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
KASSERT(rw_lock_held(&kqueue_filter_lock));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
for (i = 0; sys_kfilters[i].name != NULL; i++) {
if (strcmp(name, sys_kfilters[i].name) == 0)
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return &sys_kfilters[i];
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return NULL;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
static struct kfilter *
kfilter_byname_user(const char *name)
{
int i;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
KASSERT(rw_lock_held(&kqueue_filter_lock));
- Avoid array overrun in kfilter_byname_user() when all user kfilter slots are used: no guarantee previously that last slot had a NULL name. - Reuse previously deregistered user kfilter slots in kfilter_register().
2006-09-30 06:39:18 +04:00
/* user filter slots have a NULL name if previously deregistered */
for (i = 0; i < user_kfilterc ; i++) {
if (user_kfilters[i].name != NULL &&
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
strcmp(name, user_kfilters[i].name) == 0)
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return &user_kfilters[i];
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return NULL;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static struct kfilter *
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kfilter_byname(const char *name)
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct kfilter *kfilter;
KASSERT(rw_lock_held(&kqueue_filter_lock));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if ((kfilter = kfilter_byname_sys(name)) != NULL)
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return kfilter;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return kfilter_byname_user(name);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
* Find kfilter entry by filter id, or NULL if not found.
* Assumes entries are indexed in filter id order, for speed.
*/
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static struct kfilter *
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kfilter_byfilter(uint32_t filter)
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct kfilter *kfilter;
KASSERT(rw_lock_held(&kqueue_filter_lock));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (filter < EVFILT_SYSCOUNT) /* it's a system filter */
kfilter = &sys_kfilters[filter];
else if (user_kfilters != NULL &&
filter < EVFILT_SYSCOUNT + user_kfilterc)
/* it's a user filter */
kfilter = &user_kfilters[filter - EVFILT_SYSCOUNT];
else
return (NULL); /* out of range */
KASSERT(kfilter->filter == filter); /* sanity check! */
return (kfilter);
}
/*
* Register a new kfilter. Stores the entry in user_kfilters.
* Returns 0 if operation succeeded, or an appropriate errno(2) otherwise.
* If retfilter != NULL, the new filterid is returned in it.
*/
int
kfilter_register(const char *name, const struct filterops *filtops,
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
int *retfilter)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
struct kfilter *kfilter;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
size_t len;
- Avoid array overrun in kfilter_byname_user() when all user kfilter slots are used: no guarantee previously that last slot had a NULL name. - Reuse previously deregistered user kfilter slots in kfilter_register().
2006-09-30 06:39:18 +04:00
int i;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (name == NULL || name[0] == '\0' || filtops == NULL)
return (EINVAL); /* invalid args */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_enter(&kqueue_filter_lock, RW_WRITER);
if (kfilter_byname(name) != NULL) {
rw_exit(&kqueue_filter_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (EEXIST); /* already exists */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
if (user_kfilterc > 0xffffffff - EVFILT_SYSCOUNT) {
rw_exit(&kqueue_filter_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (EINVAL); /* too many */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
- Avoid array overrun in kfilter_byname_user() when all user kfilter slots are used: no guarantee previously that last slot had a NULL name. - Reuse previously deregistered user kfilter slots in kfilter_register().
2006-09-30 06:39:18 +04:00
for (i = 0; i < user_kfilterc; i++) {
kfilter = &user_kfilters[i];
if (kfilter->name == NULL) {
/* Previously deregistered slot. Reuse. */
goto reuse;
}
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/* check if need to grow user_kfilters */
if (user_kfilterc + 1 > user_kfiltermaxc) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
/* Grow in KFILTER_EXTENT chunks. */
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
user_kfiltermaxc += KFILTER_EXTENT;
Use sizeof correct type, not pointer to wrong type. Fixes PR/42498. This has been wrong since the initial import!
2009-12-22 23:50:46 +03:00
len = user_kfiltermaxc * sizeof(*kfilter);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kfilter = kmem_alloc(len, KM_SLEEP);
memset((char *)kfilter + user_kfiltersz, 0, len - user_kfiltersz);
if (user_kfilters != NULL) {
memcpy(kfilter, user_kfilters, user_kfiltersz);
kmem_free(user_kfilters, user_kfiltersz);
}
user_kfiltersz = len;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
user_kfilters = kfilter;
}
- Avoid array overrun in kfilter_byname_user() when all user kfilter slots are used: no guarantee previously that last slot had a NULL name. - Reuse previously deregistered user kfilter slots in kfilter_register().
2006-09-30 06:39:18 +04:00
/* Adding new slot */
kfilter = &user_kfilters[user_kfilterc++];
reuse:
Add two utility functions to help use kmem with strings: kmem_strdupsize, kmem_strfree.
2017-11-07 21:35:57 +03:00
kfilter->name = kmem_strdupsize(name, &kfilter->namelen, KM_SLEEP);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
- Avoid array overrun in kfilter_byname_user() when all user kfilter slots are used: no guarantee previously that last slot had a NULL name. - Reuse previously deregistered user kfilter slots in kfilter_register().
2006-09-30 06:39:18 +04:00
kfilter->filter = (kfilter - user_kfilters) + EVFILT_SYSCOUNT;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kfilter->filtops = kmem_alloc(sizeof(*filtops), KM_SLEEP);
memcpy(__UNCONST(kfilter->filtops), filtops, sizeof(*filtops));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (retfilter != NULL)
- Avoid array overrun in kfilter_byname_user() when all user kfilter slots are used: no guarantee previously that last slot had a NULL name. - Reuse previously deregistered user kfilter slots in kfilter_register().
2006-09-30 06:39:18 +04:00
*retfilter = kfilter->filter;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_exit(&kqueue_filter_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (0);
}
/*
* Unregister a kfilter previously registered with kfilter_register.
* This retains the filter id, but clears the name and frees filtops (filter
* operations), so that the number isn't reused during a boot.
* Returns 0 if operation succeeded, or an appropriate errno(2) otherwise.
*/
int
kfilter_unregister(const char *name)
{
struct kfilter *kfilter;
if (name == NULL || name[0] == '\0')
return (EINVAL); /* invalid name */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_enter(&kqueue_filter_lock, RW_WRITER);
if (kfilter_byname_sys(name) != NULL) {
rw_exit(&kqueue_filter_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (EINVAL); /* can't detach system filters */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kfilter = kfilter_byname_user(name);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (kfilter == NULL) {
rw_exit(&kqueue_filter_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (ENOENT);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
if (kfilter->refcnt != 0) {
rw_exit(&kqueue_filter_lock);
return (EBUSY);
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
/* Cast away const (but we know it's safe. */
kmem_free(__UNCONST(kfilter->name), kfilter->namelen);
- Avoid array overrun in kfilter_byname_user() when all user kfilter slots are used: no guarantee previously that last slot had a NULL name. - Reuse previously deregistered user kfilter slots in kfilter_register().
2006-09-30 06:39:18 +04:00
kfilter->name = NULL; /* mark as `not implemented' */
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (kfilter->filtops != NULL) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
/* Cast away const (but we know it's safe. */
kmem_free(__UNCONST(kfilter->filtops),
sizeof(*kfilter->filtops));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kfilter->filtops = NULL; /* mark as `not implemented' */
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_exit(&kqueue_filter_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (0);
}
/*
* Filter attach method for EVFILT_READ and EVFILT_WRITE on normal file
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* descriptors. Calls fileops kqfilter method for given file descriptor.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
static int
filt_fileattach(struct knote *kn)
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
file_t *fp;
fp = kn->kn_obj;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return (*fp->f_ops->fo_kqfilter)(fp, kn);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
* Filter detach method for EVFILT_READ on kqueue descriptor.
*/
static void
filt_kqdetach(struct knote *kn)
{
struct kqueue *kq;
Try not to use f_data, use f_{vnode,socket,pipe,mqueue,kqueue,ksem} to get a correctly typed pointer.
2014-09-05 13:20:59 +04:00
kq = ((file_t *)kn->kn_obj)->f_kqueue;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_spin_enter(&kq->kq_lock);
Use sel{record,remove}_knote().
2020-12-11 06:00:09 +03:00
selremove_knote(&kq->kq_sel, kn);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_spin_exit(&kq->kq_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
* Filter event method for EVFILT_READ on kqueue descriptor.
*/
/*ARGSUSED*/
static int
remove some __unused from function parameters.
2006-11-01 13:17:58 +03:00
filt_kqueue(struct knote *kn, long hint)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
struct kqueue *kq;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
int rv;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Try not to use f_data, use f_{vnode,socket,pipe,mqueue,kqueue,ksem} to get a correctly typed pointer.
2014-09-05 13:20:59 +04:00
kq = ((file_t *)kn->kn_obj)->f_kqueue;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (hint != NOTE_SUBMIT)
mutex_spin_enter(&kq->kq_lock);
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
kn->kn_data = KQ_COUNT(kq);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rv = (kn->kn_data > 0);
if (hint != NOTE_SUBMIT)
mutex_spin_exit(&kq->kq_lock);
return rv;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
* Filter attach method for EVFILT_PROC.
*/
static int
filt_procattach(struct knote *kn)
{
remove unused variable
2012-11-18 22:36:01 +04:00
struct proc *p;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Move proc_lock into the data segment. It was dynamically allocated because at the time we had mutex_obj_alloc() but not __cacheline_aligned.
2020-05-24 02:42:41 +03:00
mutex_enter(&proc_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
p = proc_find(kn->kn_id);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (p == NULL) {
Move proc_lock into the data segment. It was dynamically allocated because at the time we had mutex_obj_alloc() but not __cacheline_aligned.
2020-05-24 02:42:41 +03:00
mutex_exit(&proc_lock);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return ESRCH;
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
* Fail if it's not owned by you, or the last exec gave us
* setuid/setgid privs (unless you're root).
*/
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since we no longer need to guard against access from hardware interrupt handlers. Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the child process share the parent's lock so that signal state may be kept in sync. Partially addresses PR kern/37437.
2008-04-24 22:39:20 +04:00
mutex_enter(p->p_lock);
Move proc_lock into the data segment. It was dynamically allocated because at the time we had mutex_obj_alloc() but not __cacheline_aligned.
2020-05-24 02:42:41 +03:00
mutex_exit(&proc_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
if (kauth_authorize_process(curlwp->l_cred,
don't opencode kauth_cred_get()
2021-09-21 17:49:01 +03:00
KAUTH_PROCESS_KEVENT_FILTER, p, NULL, NULL, NULL) != 0) {
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since we no longer need to guard against access from hardware interrupt handlers. Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the child process share the parent's lock so that signal state may be kept in sync. Partially addresses PR kern/37437.
2008-04-24 22:39:20 +04:00
mutex_exit(p->p_lock);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return EACCES;
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kn->kn_obj = p;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kn->kn_flags |= EV_CLEAR; /* automatically set */
/*
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
* NOTE_CHILD is only ever generated internally; don't let it
* leak in from user-space. See knote_proc_fork_track().
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
kn->kn_sfflags &= ~NOTE_CHILD;
Add inline functions to manipulate the klists that link up knotes via kn_selnext: - klist_init() - klist_fini() - klist_insert() - klist_remove() These provide some API insulation from the implementation details of these lists (but not completely; see vn_knote_attach() and vn_knote_detach()). Currently just a wrapper around SLIST(9). This will make it significantly easier to switch kn_selnext linkage to a different kind of list.
2022-02-12 18:51:28 +03:00
klist_insert(&p->p_klist, kn);
Merge proc::p_mutex and proc::p_smutex into a single adaptive mutex, since we no longer need to guard against access from hardware interrupt handlers. Additionally, if cloning a process with CLONE_SIGHAND, arrange to have the child process share the parent's lock so that signal state may be kept in sync. Partially addresses PR kern/37437.
2008-04-24 22:39:20 +04:00
mutex_exit(p->p_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return 0;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
* Filter detach method for EVFILT_PROC.
*
* The knote may be attached to a different process, which may exit,
* leaving nothing for the knote to be attached to. So when the process
* exits, the knote is marked as DETACHED and also flagged as ONESHOT so
* it will be deleted when read out. However, as part of the knote deletion,
* this routine is called, so a check is needed to avoid actually performing
* a detach, because the original process might not exist any more.
*/
static void
filt_procdetach(struct knote *kn)
{
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
struct kqueue *kq = kn->kn_kq;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
struct proc *p;
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
/*
* We have to synchronize with knote_proc_exit(), but we
* are forced to acquire the locks in the wrong order here
* because we can't be sure kn->kn_obj is valid unless
* KN_DETACHED is not set.
*/
again:
mutex_spin_enter(&kq->kq_lock);
if ((kn->kn_status & KN_DETACHED) == 0) {
p = kn->kn_obj;
if (!mutex_tryenter(p->p_lock)) {
mutex_spin_exit(&kq->kq_lock);
preempt_point();
goto again;
}
kn->kn_status |= KN_DETACHED;
Add inline functions to manipulate the klists that link up knotes via kn_selnext: - klist_init() - klist_fini() - klist_insert() - klist_remove() These provide some API insulation from the implementation details of these lists (but not completely; see vn_knote_attach() and vn_knote_detach()). Currently just a wrapper around SLIST(9). This will make it significantly easier to switch kn_selnext linkage to a different kind of list.
2022-02-12 18:51:28 +03:00
klist_remove(&p->p_klist, kn);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
mutex_exit(p->p_lock);
}
mutex_spin_exit(&kq->kq_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
* Filter event method for EVFILT_PROC.
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
*
* Due to some of the complexities of process locking, we have special
* entry points for delivering knote submissions. filt_proc() is used
* only to check for activation from kqueue_register() and kqueue_scan().
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
static int
filt_proc(struct knote *kn, long hint)
{
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
struct kqueue *kq = kn->kn_kq;
uint32_t fflags;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
/*
* Because we share the same klist with signal knotes, just
* ensure that we're not being invoked for the proc-related
* submissions.
*/
KASSERT((hint & (NOTE_EXEC | NOTE_EXIT | NOTE_FORK)) == 0);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
mutex_spin_enter(&kq->kq_lock);
fflags = kn->kn_fflags;
mutex_spin_exit(&kq->kq_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
return fflags != 0;
}
put the exit code of the process in data, like FreeBSD does.
2015-03-02 22:24:53 +03:00
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
void
knote_proc_exec(struct proc *p)
{
struct knote *kn, *tmpkn;
struct kqueue *kq;
uint32_t fflags;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
mutex_enter(p->p_lock);
SLIST_FOREACH_SAFE(kn, &p->p_klist, kn_selnext, tmpkn) {
/* N.B. EVFILT_SIGNAL knotes are on this same list. */
if (kn->kn_fop == &sig_filtops) {
continue;
}
KASSERT(kn->kn_fop == &proc_filtops);
kq = kn->kn_kq;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_spin_enter(&kq->kq_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
fflags = (kn->kn_fflags |= (kn->kn_sfflags & NOTE_EXEC));
if (fflags) {
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
knote_activate_locked(kn);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
mutex_spin_exit(&kq->kq_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
mutex_exit(p->p_lock);
}
static int __noinline
knote_proc_fork_track(struct proc *p1, struct proc *p2, struct knote *okn)
{
struct kqueue *kq = okn->kn_kq;
KASSERT(mutex_owned(&kq->kq_lock));
KASSERT(mutex_owned(p1->p_lock));
/*
* We're going to put this knote into flux while we drop
* the locks and create and attach a new knote to track the
* child. If we are not able to enter flux, then this knote
* is about to go away, so skip the notification.
*/
if (!kn_enter_flux(okn)) {
return 0;
}
mutex_spin_exit(&kq->kq_lock);
mutex_exit(p1->p_lock);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
/*
* We actually have to register *two* new knotes:
*
* ==> One for the NOTE_CHILD notification. This is a forced
* ONESHOT note.
*
* ==> One to actually track the child process as it subsequently
* forks, execs, and, ultimately, exits.
*
* If we only register a single knote, then it's possible for
* for the NOTE_CHILD and NOTE_EXIT to be collapsed into a single
* notification if the child exits before the tracking process
* has received the NOTE_CHILD notification, which applications
* aren't expecting (the event's 'data' field would be clobbered,
fix various typos in comment, documentation and log messages.
2022-05-24 23:50:17 +03:00
* for example).
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
*
* To do this, what we have here is an **extremely** stripped-down
* version of kqueue_register() that has the following properties:
*
* ==> Does not block to allocate memory. If we are unable
* to allocate memory, we return ENOMEM.
*
* ==> Does not search for existing knotes; we know there
* are not any because this is a new process that isn't
* even visible to other processes yet.
*
* ==> Assumes that the knhash for our kq's descriptor table
* already exists (after all, we're already tracking
* processes with knotes if we got here).
*
* ==> Directly attaches the new tracking knote to the child
* process.
*
* The whole point is to do the minimum amount of work while the
* knote is held in-flux, and to avoid doing extra work in general
* (we already have the new child process; why bother looking it
* up again?).
*/
filedesc_t *fdp = kq->kq_fdp;
struct knote *knchild, *kntrack;
int error = 0;
Funnel knote alloc/free into a single pair of functions. NFCI.
2022-07-13 06:23:07 +03:00
knchild = knote_alloc(false);
kntrack = knote_alloc(false);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
if (__predict_false(knchild == NULL || kntrack == NULL)) {
error = ENOMEM;
goto out;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
kntrack->kn_obj = p2;
kntrack->kn_id = p2->p_pid;
kntrack->kn_kq = kq;
kntrack->kn_fop = okn->kn_fop;
kntrack->kn_kfilter = okn->kn_kfilter;
kntrack->kn_sfflags = okn->kn_sfflags;
kntrack->kn_sdata = p1->p_pid;
kntrack->kn_kevent.ident = p2->p_pid;
kntrack->kn_kevent.filter = okn->kn_filter;
kntrack->kn_kevent.flags =
okn->kn_flags | EV_ADD | EV_ENABLE | EV_CLEAR;
kntrack->kn_kevent.fflags = 0;
kntrack->kn_kevent.data = 0;
kntrack->kn_kevent.udata = okn->kn_kevent.udata; /* preserve udata */
/*
* The child note does not need to be attached to the
* new proc's klist at all.
*/
*knchild = *kntrack;
knchild->kn_status = KN_DETACHED;
knchild->kn_sfflags = 0;
knchild->kn_kevent.flags |= EV_ONESHOT;
knchild->kn_kevent.fflags = NOTE_CHILD;
knchild->kn_kevent.data = p1->p_pid; /* parent */
mutex_enter(&fdp->fd_lock);
/*
* We need to check to see if the kq is closing, and skip
* attaching the knote if so. Normally, this isn't necessary
* when coming in the front door because the file descriptor
* layer will synchronize this.
*
* It's safe to test KQ_CLOSING without taking the kq_lock
* here because that flag is only ever set when the fd_lock
* is also held.
*/
if (__predict_false(kq->kq_count & KQ_CLOSING)) {
mutex_exit(&fdp->fd_lock);
goto out;
}
/*
* We do the "insert into FD table" and "attach to klist" steps
* in the opposite order of kqueue_register() here to avoid
* having to take p2->p_lock twice. But this is OK because we
* hold fd_lock across the entire operation.
*/
mutex_enter(p2->p_lock);
error = kauth_authorize_process(curlwp->l_cred,
KAUTH_PROCESS_KEVENT_FILTER, p2, NULL, NULL, NULL);
if (__predict_false(error != 0)) {
mutex_exit(p2->p_lock);
mutex_exit(&fdp->fd_lock);
error = EACCES;
goto out;
}
Add inline functions to manipulate the klists that link up knotes via kn_selnext: - klist_init() - klist_fini() - klist_insert() - klist_remove() These provide some API insulation from the implementation details of these lists (but not completely; see vn_knote_attach() and vn_knote_detach()). Currently just a wrapper around SLIST(9). This will make it significantly easier to switch kn_selnext linkage to a different kind of list.
2022-02-12 18:51:28 +03:00
klist_insert(&p2->p_klist, kntrack);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
mutex_exit(p2->p_lock);
KASSERT(fdp->fd_knhashmask != 0);
KASSERT(fdp->fd_knhash != NULL);
struct klist *list = &fdp->fd_knhash[KN_HASH(kntrack->kn_id,
fdp->fd_knhashmask)];
SLIST_INSERT_HEAD(list, kntrack, kn_link);
SLIST_INSERT_HEAD(list, knchild, kn_link);
/* This adds references for knchild *and* kntrack. */
atomic_add_int(&kntrack->kn_kfilter->refcnt, 2);
knote_activate(knchild);
kntrack = NULL;
knchild = NULL;
mutex_exit(&fdp->fd_lock);
out:
if (__predict_false(knchild != NULL)) {
Funnel knote alloc/free into a single pair of functions. NFCI.
2022-07-13 06:23:07 +03:00
knote_free(knchild);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
if (__predict_false(kntrack != NULL)) {
Funnel knote alloc/free into a single pair of functions. NFCI.
2022-07-13 06:23:07 +03:00
knote_free(kntrack);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
mutex_enter(p1->p_lock);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_spin_enter(&kq->kq_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
if (kn_leave_flux(okn)) {
KQ_FLUX_WAKEUP(kq);
}
return error;
}
void
knote_proc_fork(struct proc *p1, struct proc *p2)
{
struct knote *kn;
struct kqueue *kq;
uint32_t fflags;
mutex_enter(p1->p_lock);
/*
* N.B. We DO NOT use SLIST_FOREACH_SAFE() here because we
* don't want to pre-fetch the next knote; in the event we
* have to drop p_lock, we will have put the knote in-flux,
* meaning that no one will be able to detach it until we
* have taken the knote out of flux. However, that does
* NOT stop someone else from detaching the next note in the
* list while we have it unlocked. Thus, we want to fetch
* the next note in the list only after we have re-acquired
* the lock, and using SLIST_FOREACH() will satisfy that.
*/
SLIST_FOREACH(kn, &p1->p_klist, kn_selnext) {
/* N.B. EVFILT_SIGNAL knotes are on this same list. */
if (kn->kn_fop == &sig_filtops) {
continue;
}
KASSERT(kn->kn_fop == &proc_filtops);
kq = kn->kn_kq;
mutex_spin_enter(&kq->kq_lock);
kn->kn_fflags |= (kn->kn_sfflags & NOTE_FORK);
if (__predict_false(kn->kn_sfflags & NOTE_TRACK)) {
/*
* This will drop kq_lock and p_lock and
* re-acquire them before it returns.
*/
if (knote_proc_fork_track(p1, p2, kn)) {
kn->kn_fflags |= NOTE_TRACKERR;
}
KASSERT(mutex_owned(p1->p_lock));
KASSERT(mutex_owned(&kq->kq_lock));
}
fflags = kn->kn_fflags;
if (fflags) {
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
knote_activate_locked(kn);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
mutex_spin_exit(&kq->kq_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
mutex_exit(p1->p_lock);
}
void
knote_proc_exit(struct proc *p)
{
struct knote *kn;
struct kqueue *kq;
KASSERT(mutex_owned(p->p_lock));
while (!SLIST_EMPTY(&p->p_klist)) {
kn = SLIST_FIRST(&p->p_klist);
kq = kn->kn_kq;
KASSERT(kn->kn_obj == p);
mutex_spin_enter(&kq->kq_lock);
kn->kn_data = P_WAITSTATUS(p);
/*
* Mark as ONESHOT, so that the knote is g/c'ed
* when read.
*/
kn->kn_flags |= (EV_EOF | EV_ONESHOT);
kn->kn_fflags |= kn->kn_sfflags & NOTE_EXIT;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
* Detach the knote from the process and mark it as such.
* N.B. EVFILT_SIGNAL are also on p_klist, but by the
* time we get here, all open file descriptors for this
* process have been released, meaning that signal knotes
* will have already been detached.
*
* We need to synchronize this with filt_procdetach().
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
KASSERT(kn->kn_fop == &proc_filtops);
if ((kn->kn_status & KN_DETACHED) == 0) {
kn->kn_status |= KN_DETACHED;
SLIST_REMOVE_HEAD(&p->p_klist, kn_selnext);
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
/*
* Always activate the knote for NOTE_EXIT regardless
* of whether or not the listener cares about it.
* This matches historical behavior.
*/
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
knote_activate_locked(kn);
mutex_spin_exit(&kq->kq_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
#define FILT_TIMER_NOSCHED ((uintptr_t)-1)
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
static int
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
filt_timercompute(struct kevent *kev, uintptr_t *tticksp)
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
{
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
struct timespec ts;
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
uintptr_t tticks;
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
if (kev->fflags & ~(NOTE_TIMER_UNITMASK | NOTE_ABSTIME)) {
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
return EINVAL;
}
/*
* Convert the event 'data' to a timespec, then convert the
* timespec to callout ticks.
*/
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
switch (kev->fflags & NOTE_TIMER_UNITMASK) {
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
case NOTE_SECONDS:
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
ts.tv_sec = kev->data;
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
ts.tv_nsec = 0;
break;
case NOTE_MSECONDS: /* == historical value 0 */
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
ts.tv_sec = kev->data / 1000;
ts.tv_nsec = (kev->data % 1000) * 1000000;
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
break;
case NOTE_USECONDS:
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
ts.tv_sec = kev->data / 1000000;
ts.tv_nsec = (kev->data % 1000000) * 1000;
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
break;
case NOTE_NSECONDS:
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
ts.tv_sec = kev->data / 1000000000;
ts.tv_nsec = kev->data % 1000000000;
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
break;
default:
return EINVAL;
}
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
if (kev->fflags & NOTE_ABSTIME) {
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
struct timespec deadline = ts;
/*
* Get current time.
*
* XXX This is CLOCK_REALTIME. There is no way to
* XXX specify CLOCK_MONOTONIC.
*/
nanotime(&ts);
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
/* Absolute timers do not repeat. */
kev->data = FILT_TIMER_NOSCHED;
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
/* If we're past the deadline, then the event will fire. */
if (timespeccmp(&deadline, &ts, <=)) {
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
tticks = FILT_TIMER_NOSCHED;
goto out;
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
}
/* Calculate how much time is left. */
timespecsub(&deadline, &ts, &ts);
} else {
/* EV_CLEAR automatically set for relative timers. */
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
kev->flags |= EV_CLEAR;
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
}
tticks = tstohz(&ts);
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
/* if the supplied value is under our resolution, use 1 tick */
if (tticks == 0) {
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
if (kev->data == 0)
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return EINVAL;
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
tticks = 1;
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
} else if (tticks > INT_MAX) {
return EINVAL;
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
}
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
if ((kev->flags & EV_ONESHOT) != 0) {
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
/* Timer does not repeat. */
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
kev->data = FILT_TIMER_NOSCHED;
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
} else {
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
KASSERT((uintptr_t)tticks != FILT_TIMER_NOSCHED);
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
kev->data = tticks;
}
out:
*tticksp = tticks;
return 0;
}
static void
filt_timerexpire(void *knx)
{
struct knote *kn = knx;
struct kqueue *kq = kn->kn_kq;
mutex_spin_enter(&kq->kq_lock);
kn->kn_data++;
knote_activate_locked(kn);
if (kn->kn_sdata != FILT_TIMER_NOSCHED) {
kern: KASSERT(A && B) -> KASSERT(A); KASSERT(B)
2023-04-09 12:18:09 +03:00
KASSERT(kn->kn_sdata > 0);
KASSERT(kn->kn_sdata <= INT_MAX);
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
callout_schedule((callout_t *)kn->kn_hook,
(int)kn->kn_sdata);
}
mutex_spin_exit(&kq->kq_lock);
}
Support modifying an existing timer without having to delete it first. Semantics match FreeBSD.
2021-10-22 07:49:24 +03:00
static inline void
filt_timerstart(struct knote *kn, uintptr_t tticks)
{
callout_t *calloutp = kn->kn_hook;
KASSERT(mutex_owned(&kn->kn_kq->kq_lock));
KASSERT(!callout_pending(calloutp));
if (__predict_false(tticks == FILT_TIMER_NOSCHED)) {
kn->kn_data = 1;
} else {
KASSERT(tticks <= INT_MAX);
callout_reset(calloutp, (int)tticks, filt_timerexpire, kn);
}
}
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
static int
filt_timerattach(struct knote *kn)
{
callout_t *calloutp;
struct kqueue *kq;
uintptr_t tticks;
int error;
struct kevent kev = {
.flags = kn->kn_flags,
.fflags = kn->kn_sfflags,
.data = kn->kn_sdata,
};
error = filt_timercompute(&kev, &tticks);
if (error) {
return error;
Add support for the NOTE_SECONDS, NOTE_MSECONDS, NOTE_USECONDS, NOTE_NSECONDS, and NOTE_ABSTIME filter flags to EVFILT_TIMER, API-compatible with the same in FreeBSD.
2021-10-13 07:57:19 +03:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (atomic_inc_uint_nv(&kq_ncallouts) >= kq_calloutmax ||
(calloutp = kmem_alloc(sizeof(*calloutp), KM_NOSLEEP)) == NULL) {
atomic_dec_uint(&kq_ncallouts);
return ENOMEM;
}
Mark the callout MPSAFE and use callout_halt().
2008-04-22 15:44:24 +04:00
callout_init(calloutp, CALLOUT_MPSAFE);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kq = kn->kn_kq;
mutex_spin_enter(&kq->kq_lock);
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
kn->kn_sdata = kev.data;
kn->kn_flags = kev.flags;
KASSERT(kn->kn_sfflags == kev.fflags);
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
kn->kn_hook = calloutp;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
Support modifying an existing timer without having to delete it first. Semantics match FreeBSD.
2021-10-22 07:49:24 +03:00
filt_timerstart(kn, tticks);
Re-factor the code that computes the EVFILT_TIMER value into its own function. NFC.
2021-10-21 04:11:21 +03:00
mutex_spin_exit(&kq->kq_lock);
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
return (0);
}
static void
filt_timerdetach(struct knote *kn)
{
Merge some of the less invasive changes from the vmlocking branch: - kthread, callout, devsw API changes - select()/poll() improvements - miscellaneous MT safety improvements
2007-07-10 00:51:58 +04:00
callout_t *calloutp;
Set EV_ONESHOT to prevent rescheduling XXX: pullup-8
2018-01-12 20:58:51 +03:00
struct kqueue *kq = kn->kn_kq;
/* prevent rescheduling when we expire */
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
mutex_spin_enter(&kq->kq_lock);
kn->kn_sdata = FILT_TIMER_NOSCHED;
Set EV_ONESHOT to prevent rescheduling XXX: pullup-8
2018-01-12 20:58:51 +03:00
mutex_spin_exit(&kq->kq_lock);
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
Merge some of the less invasive changes from the vmlocking branch: - kthread, callout, devsw API changes - select()/poll() improvements - miscellaneous MT safety improvements
2007-07-10 00:51:58 +04:00
calloutp = (callout_t *)kn->kn_hook;
- Rename kqueue_misc_lock -> kqueue_timer_lock, since EVFILT_TIMER is now its only user. Also initialize it as IPL_SOFTCLOCK; there is no practical difference in how it operates (it is still an adaptive lock), but this serves as a visual reminder that we are interlocking against a callout. - Add some comments that describe why we don't need to hold kqueue_timer_lock when detaching an EVFILT_TIMER due to guarantees made by callout_halt(). - Mark timer_filtops as MPSAFE.
2021-09-27 02:34:46 +03:00
/*
* Attempt to stop the callout. This will block if it's
* already running.
*/
Give callout_halt() an additional 'kmutex_t *interlock' argument. If there is a need to block and wait for the callout to complete, and there is an interlock, it will be dropped while waiting and reacquired before return.
2008-04-22 16:04:22 +04:00
callout_halt(calloutp, NULL);
- Rename kqueue_misc_lock -> kqueue_timer_lock, since EVFILT_TIMER is now its only user. Also initialize it as IPL_SOFTCLOCK; there is no practical difference in how it operates (it is still an adaptive lock), but this serves as a visual reminder that we are interlocking against a callout. - Add some comments that describe why we don't need to hold kqueue_timer_lock when detaching an EVFILT_TIMER due to guarantees made by callout_halt(). - Mark timer_filtops as MPSAFE.
2021-09-27 02:34:46 +03:00
Merge some of the less invasive changes from the vmlocking branch: - kthread, callout, devsw API changes - select()/poll() improvements - miscellaneous MT safety improvements
2007-07-10 00:51:58 +04:00
callout_destroy(calloutp);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kmem_free(calloutp, sizeof(*calloutp));
atomic_dec_uint(&kq_ncallouts);
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
}
Support modifying an existing timer without having to delete it first. Semantics match FreeBSD.
2021-10-22 07:49:24 +03:00
static int
filt_timertouch(struct knote *kn, struct kevent *kev, long type)
{
struct kqueue *kq = kn->kn_kq;
callout_t *calloutp;
uintptr_t tticks;
int error;
KASSERT(mutex_owned(&kq->kq_lock));
switch (type) {
case EVENT_REGISTER:
/* Only relevant for EV_ADD. */
if ((kev->flags & EV_ADD) == 0) {
return 0;
}
/*
* Stop the timer, under the assumption that if
* an application is re-configuring the timer,
* they no longer care about the old one. We
* can safely drop the kq_lock while we wait
* because fdp->fd_lock will be held throughout,
* ensuring that no one can sneak in with an
* EV_DELETE or close the kq.
*/
KASSERT(mutex_owned(&kq->kq_fdp->fd_lock));
calloutp = kn->kn_hook;
callout_halt(calloutp, &kq->kq_lock);
KASSERT(mutex_owned(&kq->kq_lock));
knote_deactivate_locked(kn);
kn->kn_data = 0;
error = filt_timercompute(kev, &tticks);
if (error) {
return error;
}
kn->kn_sdata = kev->data;
kn->kn_flags = kev->flags;
kn->kn_sfflags = kev->fflags;
filt_timerstart(kn, tticks);
break;
case EVENT_PROCESS:
*kev = kn->kn_kevent;
break;
default:
panic("%s: invalid type (%ld)", __func__, type);
}
return 0;
}
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
static int
remove some __unused from function parameters.
2006-11-01 13:17:58 +03:00
filt_timer(struct knote *kn, long hint)
Introduce EVFILT_TIMER, which allows a process to establish an arbitrary number of timers, both oneshot and periodic. from FreeBSD, only adapted to NetBSD kernel API - mstohz() instead of tvtohz(), and takes advantage of callout_schedule() in filt_timerexpire()
2003-02-04 12:02:04 +03:00
{
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
struct kqueue *kq = kn->kn_kq;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
int rv;
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
mutex_spin_enter(&kq->kq_lock);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rv = (kn->kn_data != 0);
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
mutex_spin_exit(&kq->kq_lock);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
Merge autofs support from: Tomohiro Kusumi XXX: Does not work yet
2018-01-09 06:31:12 +03:00
return rv;
}
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
static int
filt_userattach(struct knote *kn)
{
struct kqueue *kq = kn->kn_kq;
/*
* EVFILT_USER knotes are not attached to anything in the kernel.
*/
mutex_spin_enter(&kq->kq_lock);
kn->kn_hook = NULL;
if (kn->kn_fflags & NOTE_TRIGGER)
kn->kn_hookid = 1;
else
kn->kn_hookid = 0;
mutex_spin_exit(&kq->kq_lock);
return (0);
}
static void
filt_userdetach(struct knote *kn)
{
/*
* EVFILT_USER knotes are not attached to anything in the kernel.
*/
}
static int
filt_user(struct knote *kn, long hint)
{
struct kqueue *kq = kn->kn_kq;
int hookid;
mutex_spin_enter(&kq->kq_lock);
hookid = kn->kn_hookid;
mutex_spin_exit(&kq->kq_lock);
return hookid;
}
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
static int
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
filt_usertouch(struct knote *kn, struct kevent *kev, long type)
{
int ffctrl;
Fix non-DIAGNOSTIC build
2021-01-27 09:59:08 +03:00
KASSERT(mutex_owned(&kn->kn_kq->kq_lock));
call f_touch with kq_lock held, and without KERNEL_LOCK() - for this adjust EVFILT_USER, which is the only filter actually using that hook kqueue_scan() now doesn't need to exit/enter the kq_lock when calling f_touch, which removes another possible race part of PR kern/50094
2021-01-26 22:09:18 +03:00
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
switch (type) {
case EVENT_REGISTER:
if (kev->fflags & NOTE_TRIGGER)
kn->kn_hookid = 1;
ffctrl = kev->fflags & NOTE_FFCTRLMASK;
kev->fflags &= NOTE_FFLAGSMASK;
switch (ffctrl) {
case NOTE_FFNOP:
break;
case NOTE_FFAND:
kn->kn_sfflags &= kev->fflags;
break;
case NOTE_FFOR:
kn->kn_sfflags |= kev->fflags;
break;
case NOTE_FFCOPY:
kn->kn_sfflags = kev->fflags;
break;
default:
/* XXX Return error? */
break;
}
kn->kn_sdata = kev->data;
if (kev->flags & EV_CLEAR) {
kn->kn_hookid = 0;
kn->kn_data = 0;
kn->kn_fflags = 0;
}
break;
case EVENT_PROCESS:
*kev = kn->kn_kevent;
kev->fflags = kn->kn_sfflags;
kev->data = kn->kn_sdata;
if (kn->kn_flags & EV_CLEAR) {
kn->kn_hookid = 0;
kn->kn_data = 0;
kn->kn_fflags = 0;
}
break;
default:
panic("filt_usertouch() - invalid type (%ld)", type);
break;
}
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
return 0;
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
* filt_seltrue:
*
* This filter "event" routine simulates seltrue().
*/
int
remove some __unused from function parameters.
2006-11-01 13:17:58 +03:00
filt_seltrue(struct knote *kn, long hint)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
/*
* We don't know how much data can be read/written,
* but we know that it *can* be. This is about as
* good as select/poll does as well.
*/
kn->kn_data = 0;
return (1);
}
/*
* This provides full kqfilter entry for device switch tables, which
* has same effect as filter using filt_seltrue() as filter method.
*/
static void
remove some __unused from function parameters.
2006-11-01 13:17:58 +03:00
filt_seltruedetach(struct knote *kn)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
/* Nothing to do */
}
Use C99 initializer for filterops Mostly done with spatch with touchups for indentation @@ expression a; identifier b,c,d; identifier p; @@ const struct filterops p = - { a, b, c, d + { + .f_isfd = a, + .f_attach = b, + .f_detach = c, + .f_event = d, };
2017-10-25 11:12:37 +03:00
const struct filterops seltrue_filtops = {
Mark kqread_filtops, user_filtops, and seltrue_filtops as MPSAFE.
2021-09-26 21:13:58 +03:00
.f_flags = FILTEROP_ISFD | FILTEROP_MPSAFE,
Use C99 initializer for filterops Mostly done with spatch with touchups for indentation @@ expression a; identifier b,c,d; identifier p; @@ const struct filterops p = - { a, b, c, d + { + .f_isfd = a, + .f_attach = b, + .f_detach = c, + .f_event = d, };
2017-10-25 11:12:37 +03:00
.f_attach = NULL,
.f_detach = filt_seltruedetach,
.f_event = filt_seltrue,
};
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
int
remove some __unused from function parameters.
2006-11-01 13:17:58 +03:00
seltrue_kqfilter(dev_t dev, struct knote *kn)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
switch (kn->kn_filter) {
case EVFILT_READ:
case EVFILT_WRITE:
kn->kn_fop = &seltrue_filtops;
break;
default:
Do not "return 1" from kqfilter for errors. That value is passed directly to the userland caller and results in a mysterious EPERM. Instead, return EINVAL or something else sensible depending on the case.
2007-12-05 20:19:46 +03:00
return (EINVAL);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/* Nothing more to do */
return (0);
}
/*
* kqueue(2) system call.
*/
* Arrange for interfaces that create new file descriptors to be able to set close-on-exec on creation (http://udrepper.livejournal.com/20407.html). - Add F_DUPFD_CLOEXEC to fcntl(2). - Add MSG_CMSG_CLOEXEC to recvmsg(2) for unix file descriptor passing. - Add dup3(2) syscall with a flags argument for O_CLOEXEC, O_NONBLOCK. - Add pipe2(2) syscall with a flags argument for O_CLOEXEC, O_NONBLOCK. - Add flags SOCK_CLOEXEC, SOCK_NONBLOCK to the socket type parameter for socket(2) and socketpair(2). - Add new paccept(2) syscall that takes an additional sigset_t to alter the sigmask temporarily and a flags argument to set SOCK_CLOEXEC, SOCK_NONBLOCK. - Add new mode character 'e' to fopen(3) and popen(3) to open pipes and file descriptors for close on exec. - Add new kqueue1(2) syscall with a new flags argument to open the kqueue file descriptor with O_CLOEXEC, O_NONBLOCK. * Fix the system calls that take socklen_t arguments to actually do so. * Don't include userland header files (signal.h) from system header files (rump_syscallargs.h). * Bump libc version for the new syscalls.
2011-06-26 20:42:39 +04:00
static int
kqueue1(struct lwp *l, int flags, register_t *retval)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct kqueue *kq;
file_t *fp;
int fd, error;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if ((error = fd_allocfile(&fp, &fd)) != 0)
return error;
As discussed in tech-kern, provide the means to prevent delivery of SIGPIPE on EPIPE for all file descriptor types: - provide O_NOSIGPIPE for open,kqueue1,pipe2,dup3,fcntl(F_{G,S}ETFL) [NetBSD] - provide SOCK_NOSIGPIPE for socket,socketpair [NetBSD] - provide SO_NOSIGPIPE for {g,s}seckopt [NetBSD/FreeBSD/MacOSX] - provide F_{G,S}ETNOSIGPIPE for fcntl [MacOSX]
2012-01-25 04:28:35 +04:00
fp->f_flag = FREAD | FWRITE | (flags & (FNONBLOCK|FNOSIGPIPE));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
fp->f_type = DTYPE_KQUEUE;
fp->f_ops = &kqueueops;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kq = kmem_zalloc(sizeof(*kq), KM_SLEEP);
mutex_init(&kq->kq_lock, MUTEX_DEFAULT, IPL_SCHED);
cv_init(&kq->kq_cv, "kqueue");
Welcome to 4.99.55: - Add a lot of missing selinit() and seldestroy() calls. - Merge selwakeup() and selnotify() calls into a single selnotify(). - Add an additional 'events' argument to selnotify() call. It will indicate which event (POLL_IN, POLL_OUT, etc) happen. If unknown, zero may be used. Note: please pass appropriate value of 'events' where possible. Proposed on: <tech-kern>
2008-03-01 17:16:49 +03:00
selinit(&kq->kq_sel);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
TAILQ_INIT(&kq->kq_head);
Try not to use f_data, use f_{vnode,socket,pipe,mqueue,kqueue,ksem} to get a correctly typed pointer.
2014-09-05 13:20:59 +04:00
fp->f_kqueue = kq;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*retval = fd;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kq->kq_fdp = curlwp->l_fd;
* Arrange for interfaces that create new file descriptors to be able to set close-on-exec on creation (http://udrepper.livejournal.com/20407.html). - Add F_DUPFD_CLOEXEC to fcntl(2). - Add MSG_CMSG_CLOEXEC to recvmsg(2) for unix file descriptor passing. - Add dup3(2) syscall with a flags argument for O_CLOEXEC, O_NONBLOCK. - Add pipe2(2) syscall with a flags argument for O_CLOEXEC, O_NONBLOCK. - Add flags SOCK_CLOEXEC, SOCK_NONBLOCK to the socket type parameter for socket(2) and socketpair(2). - Add new paccept(2) syscall that takes an additional sigset_t to alter the sigmask temporarily and a flags argument to set SOCK_CLOEXEC, SOCK_NONBLOCK. - Add new mode character 'e' to fopen(3) and popen(3) to open pipes and file descriptors for close on exec. - Add new kqueue1(2) syscall with a new flags argument to open the kqueue file descriptor with O_CLOEXEC, O_NONBLOCK. * Fix the system calls that take socklen_t arguments to actually do so. * Don't include userland header files (signal.h) from system header files (rump_syscallargs.h). * Bump libc version for the new syscalls.
2011-06-26 20:42:39 +04:00
fd_set_exclose(l, fd, (flags & O_CLOEXEC) != 0);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fd_affix(curproc, fp, fd);
return error;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
* Arrange for interfaces that create new file descriptors to be able to set close-on-exec on creation (http://udrepper.livejournal.com/20407.html). - Add F_DUPFD_CLOEXEC to fcntl(2). - Add MSG_CMSG_CLOEXEC to recvmsg(2) for unix file descriptor passing. - Add dup3(2) syscall with a flags argument for O_CLOEXEC, O_NONBLOCK. - Add pipe2(2) syscall with a flags argument for O_CLOEXEC, O_NONBLOCK. - Add flags SOCK_CLOEXEC, SOCK_NONBLOCK to the socket type parameter for socket(2) and socketpair(2). - Add new paccept(2) syscall that takes an additional sigset_t to alter the sigmask temporarily and a flags argument to set SOCK_CLOEXEC, SOCK_NONBLOCK. - Add new mode character 'e' to fopen(3) and popen(3) to open pipes and file descriptors for close on exec. - Add new kqueue1(2) syscall with a new flags argument to open the kqueue file descriptor with O_CLOEXEC, O_NONBLOCK. * Fix the system calls that take socklen_t arguments to actually do so. * Don't include userland header files (signal.h) from system header files (rump_syscallargs.h). * Bump libc version for the new syscalls.
2011-06-26 20:42:39 +04:00
/*
* kqueue(2) system call.
*/
int
sys_kqueue(struct lwp *l, const void *v, register_t *retval)
{
return kqueue1(l, 0, retval);
}
int
sys_kqueue1(struct lwp *l, const struct sys_kqueue1_args *uap,
register_t *retval)
{
/* {
syscallarg(int) flags;
} */
return kqueue1(l, SCARG(uap, flags), retval);
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
* kevent(2) system call.
*/
merge christos-time_t
2009-01-11 05:45:45 +03:00
int
Don't next structure and enum definitions. Don't use C++ keywords new, try, class, private, etc.
2014-09-05 09:57:21 +04:00
kevent_fetch_changes(void *ctx, const struct kevent *changelist,
merge christos-time_t
2009-01-11 05:45:45 +03:00
struct kevent *changes, size_t index, int n)
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
return copyin(changelist + index, changes, n * sizeof(*changes));
}
merge christos-time_t
2009-01-11 05:45:45 +03:00
int
Don't next structure and enum definitions. Don't use C++ keywords new, try, class, private, etc.
2014-09-05 09:57:21 +04:00
kevent_put_events(void *ctx, struct kevent *events,
merge christos-time_t
2009-01-11 05:45:45 +03:00
struct kevent *eventlist, size_t index, int n)
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
return copyout(events, eventlist + index, n * sizeof(*events));
}
static const struct kevent_ops kevent_native_ops = {
Replace gcc-style designated initialisers with c99-style.
2008-06-24 14:27:35 +04:00
.keo_private = NULL,
.keo_fetch_timeout = copyin,
.keo_fetch_changes = kevent_fetch_changes,
.keo_put_events = kevent_put_events,
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
};
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
int
Add epoll(2) from Theodore Preduta as part of GSoC 2023
2023-07-28 21:18:59 +03:00
sys___kevent100(struct lwp *l, const struct sys___kevent100_args *uap,
merge christos-time_t
2009-01-11 05:45:45 +03:00
register_t *retval)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
Convert all the system call entry points from: int foo(struct lwp *l, void *v, register_t *retval) to: int foo(struct lwp *l, const struct foo_args *uap, register_t *retval) Fixup compat code to not write into 'uap' and (in some cases) to actually pass a correctly formatted 'uap' structure with the right name to the next routine. A few 'compat' routines that just call standard ones have been deleted. All the 'compat' code compiles (along with the kernels required to test build it). 98% done by automated scripts.
2007-12-21 02:02:38 +03:00
/* {
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
syscallarg(int) fd;
syscallarg(const struct kevent *) changelist;
syscallarg(size_t) nchanges;
syscallarg(struct kevent *) eventlist;
syscallarg(size_t) nevents;
syscallarg(const struct timespec *) timeout;
Convert all the system call entry points from: int foo(struct lwp *l, void *v, register_t *retval) to: int foo(struct lwp *l, const struct foo_args *uap, register_t *retval) Fixup compat code to not write into 'uap' and (in some cases) to actually pass a correctly formatted 'uap' structure with the right name to the next routine. A few 'compat' routines that just call standard ones have been deleted. All the 'compat' code compiles (along with the kernels required to test build it). 98% done by automated scripts.
2007-12-21 02:02:38 +03:00
} */
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return kevent1(retval, SCARG(uap, fd), SCARG(uap, changelist),
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
SCARG(uap, nchanges), SCARG(uap, eventlist), SCARG(uap, nevents),
SCARG(uap, timeout), &kevent_native_ops);
}
int
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kevent1(register_t *retval, int fd,
const struct kevent *changelist, size_t nchanges,
struct kevent *eventlist, size_t nevents,
const struct timespec *timeout,
const struct kevent_ops *keops)
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct kevent *kevp;
struct kqueue *kq;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
struct timespec ts;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
size_t i, n, ichange;
int nerrors, error;
Do not hardcode the value. Use KQ_NEVENTS.
2014-06-24 18:42:43 +04:00
struct kevent kevbuf[KQ_NEVENTS]; /* approx 300 bytes on 64-bit */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
file_t *fp;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/* check that we're dealing with a kq */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fp = fd_getfile(fd);
Make updating a file's reference and use count MP-safe.
2003-02-23 17:37:32 +03:00
if (fp == NULL)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (EBADF);
Make updating a file's reference and use count MP-safe.
2003-02-23 17:37:32 +03:00
if (fp->f_type != DTYPE_KQUEUE) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fd_putfile(fd);
Make updating a file's reference and use count MP-safe.
2003-02-23 17:37:32 +03:00
return (EBADF);
}
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
if (timeout != NULL) {
error = (*keops->keo_fetch_timeout)(timeout, &ts, sizeof(ts));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (error)
goto done;
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
timeout = &ts;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
Try not to use f_data, use f_{vnode,socket,pipe,mqueue,kqueue,ksem} to get a correctly typed pointer.
2014-09-05 13:20:59 +04:00
kq = fp->f_kqueue;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
nerrors = 0;
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
ichange = 0;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/* traverse list of events to register */
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
while (nchanges > 0) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
n = MIN(nchanges, __arraycount(kevbuf));
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
error = (*keops->keo_fetch_changes)(keops->keo_private,
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
changelist, kevbuf, ichange, n);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (error)
goto done;
for (i = 0; i < n; i++) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kevp = &kevbuf[i];
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kevp->flags &= ~EV_SYSFLAGS;
/* register each knote */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
error = kqueue_register(kq, kevp);
Rearrange the if conditions in order to get rid of unnecessary indentation. No functional change intended. ok christos@
2017-04-27 19:52:22 +03:00
if (!error && !(kevp->flags & EV_RECEIPT))
continue;
if (nevents == 0)
goto done;
kevp->flags = EV_ERROR;
kevp->data = error;
error = (*keops->keo_put_events)
(keops->keo_private, kevp,
eventlist, nerrors, 1);
if (error)
goto done;
nevents--;
nerrors++;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
nchanges -= n; /* update the results */
ichange += n;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
if (nerrors) {
*retval = nerrors;
error = 0;
goto done;
}
/* actually scan through the events */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
error = kqueue_scan(fp, nevents, eventlist, timeout, retval, keops,
kevbuf, __arraycount(kevbuf));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
done:
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fd_putfile(fd);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (error);
}
/*
* Register a given kevent kev onto the kqueue
*/
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static int
kqueue_register(struct kqueue *kq, struct kevent *kev)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct kfilter *kfilter;
filedesc_t *fdp;
file_t *fp;
fdfile_t *ff;
struct knote *kn, *newkn;
struct klist *list;
int error, fd, rv;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
fdp = kq->kq_fdp;
fp = NULL;
kn = NULL;
error = 0;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fd = 0;
Funnel knote alloc/free into a single pair of functions. NFCI.
2022-07-13 06:23:07 +03:00
newkn = knote_alloc(true);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_enter(&kqueue_filter_lock, RW_READER);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kfilter = kfilter_byfilter(kev->filter);
if (kfilter == NULL || kfilter->filtops == NULL) {
/* filter not found nor implemented */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_exit(&kqueue_filter_lock);
Funnel knote alloc/free into a single pair of functions. NFCI.
2022-07-13 06:23:07 +03:00
knote_free(newkn);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (EINVAL);
}
/* search if knote already exists */
Change the kqueue filterops::f_isfd field to filterops::f_flags, and define a flag FILTEROP_ISFD that has the meaning of the prior f_isfd. Field and flag name aligned with OpenBSD. This does not constitute a functional or ABI change, as the field location and size, and the value placed in that field, are the same as the previous code, but we're bumping __NetBSD_Version__ so 3rd-party module source code can adapt, as needed. NetBSD 9.99.89
2021-09-26 04:16:07 +03:00
if (kfilter->filtops->f_flags & FILTEROP_ISFD) {
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/* monitoring a file descriptor */
From tedu at openbsd: kevent validates that ident is a valid fd by getting the file. one sad quirk: uint64 to int32 truncation can lead to false positives, and then later in the array sizing code, very big mallocs panic the kernel. add a check that the ident isn't larger than INT_MAX in the fd case. reported by Tim Newsham
2016-07-14 09:22:17 +03:00
/* validate descriptor */
make sure we cleanup properly when fd is too big.
2016-07-14 21:16:51 +03:00
if (kev->ident > INT_MAX
|| (fp = fd_getfile(fd = kev->ident)) == NULL) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_exit(&kqueue_filter_lock);
Funnel knote alloc/free into a single pair of functions. NFCI.
2022-07-13 06:23:07 +03:00
knote_free(newkn);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return EBADF;
}
kqueue_register: avoid calling fd_getfile() with filedesc_t::fd_lock held. Fixes PR/45479 by KOGULE Ryo.
2011-11-18 02:41:55 +04:00
mutex_enter(&fdp->fd_lock);
More changes to improve kern_descrip.c. - Avoid atomics in more places. - Remove the per-descriptor mutex, and just use filedesc_t::fd_lock. It was only being used to synchronize close, and in any case we needed to take fd_lock to free the descriptor slot. - Optimize certain paths for the <NDFDFILE case. - Sprinkle more comments and assertions. - Cache more stuff in filedesc_t. - Fix numerous minor bugs spotted along the way. - Restructure how the open files array is maintained, for clarity and so that we can eliminate the membar_consumer() call in fd_getfile(). This is mostly syntactic sugar; the main functional change is that fd_nfiles now lives alongside the open file array. Some measurements with libmicro: - simple file syscalls are like close() are between 1 to 10% faster. - some nice improvements, e.g. poll(1000) which is ~50% faster.
2009-05-25 01:41:25 +04:00
ff = fdp->fd_dt->dt_ff[fd];
Don't add kevents to closing file descriptors (from riastradh)
2017-11-11 06:58:01 +03:00
if (ff->ff_refcnt & FR_CLOSING) {
error = EBADF;
goto doneunlock;
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (fd <= fdp->fd_lastkqfile) {
SLIST_FOREACH(kn, &ff->ff_knlist, kn_link) {
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (kq == kn->kn_kq &&
kev->filter == kn->kn_filter)
break;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
} else {
/*
* not monitoring a file descriptor, so
* lookup knotes in internal hash table
*/
kqueue_register: avoid calling fd_getfile() with filedesc_t::fd_lock held. Fixes PR/45479 by KOGULE Ryo.
2011-11-18 02:41:55 +04:00
mutex_enter(&fdp->fd_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (fdp->fd_knhashmask != 0) {
list = &fdp->fd_knhash[
KN_HASH((u_long)kev->ident, fdp->fd_knhashmask)];
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
SLIST_FOREACH(kn, list, kn_link) {
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (kev->ident == kn->kn_id &&
kq == kn->kn_kq &&
kev->filter == kn->kn_filter)
break;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
}
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
/* It's safe to test KQ_CLOSING while holding only the fd_lock. */
KASSERT(mutex_owned(&fdp->fd_lock));
KASSERT((kq->kq_count & KQ_CLOSING) == 0);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
* kn now contains the matching knote, or NULL if no match
*/
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
if (kn == NULL) {
if (kev->flags & EV_ADD) {
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/* create new knote */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kn = newkn;
newkn = NULL;
kn->kn_obj = fp;
- initialize kn_id - in close, invalidate f_data and f_type early to prevent accidental re-use - add a DIAGNOSTIC for when we use unsupported fd's and a KASSERT for f_event being NULL.
2012-11-24 19:14:32 +04:00
kn->kn_id = kev->ident;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kn->kn_kq = kq;
kn->kn_fop = kfilter->filtops;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kn->kn_kfilter = kfilter;
kn->kn_sfflags = kev->fflags;
kn->kn_sdata = kev->data;
kev->fflags = 0;
kev->data = 0;
kn->kn_kevent = *kev;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
PR/50730: Benny Siegert: Go kqueue test panics kernel. - use a marker knote from the stack instead of allocating and freeing on each scan. - add more KASSERTS - introduce a KN_BUSY bit that indicates that the knote is currently being scanned, so that knote_detach does not end up deleting it when the file descriptor gets closed and we don't end up using/trashing free memory from the scan.
2016-01-31 07:40:01 +03:00
KASSERT(kn->kn_fop != NULL);
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
/*
* XXX Allow only known-safe users of f_touch.
* XXX See filter_touch() for details.
*/
if (kn->kn_fop->f_touch != NULL &&
kn->kn_fop != &timer_filtops &&
kn->kn_fop != &user_filtops) {
error = ENOTSUP;
goto fail_ev_add;
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
* apply reference count to knote structure, and
* do not release it at the end of this routine.
*/
fp = NULL;
Change the kqueue filterops::f_isfd field to filterops::f_flags, and define a flag FILTEROP_ISFD that has the meaning of the prior f_isfd. Field and flag name aligned with OpenBSD. This does not constitute a functional or ABI change, as the field location and size, and the value placed in that field, are the same as the previous code, but we're bumping __NetBSD_Version__ so 3rd-party module source code can adapt, as needed. NetBSD 9.99.89
2021-09-26 04:16:07 +03:00
if (!(kn->kn_fop->f_flags & FILTEROP_ISFD)) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
/*
* If knote is not on an fd, store on
* internal hash table.
*/
if (fdp->fd_knhashmask == 0) {
/* XXXAD can block with fd_lock held */
fdp->fd_knhash = hashinit(KN_HASHSIZE,
- Convert hashinit() to use kmem_alloc(). The hash tables can be large and it's better to not have them in kmem_map. - Convert a couple of minor items along the way to kmem_alloc(). - Fix some memory leaks.
2008-05-05 21:11:16 +04:00
HASH_LIST, true,
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
&fdp->fd_knhashmask);
}
list = &fdp->fd_knhash[KN_HASH(kn->kn_id,
fdp->fd_knhashmask)];
} else {
/* Otherwise, knote is on an fd. */
list = (struct klist *)
More changes to improve kern_descrip.c. - Avoid atomics in more places. - Remove the per-descriptor mutex, and just use filedesc_t::fd_lock. It was only being used to synchronize close, and in any case we needed to take fd_lock to free the descriptor slot. - Optimize certain paths for the <NDFDFILE case. - Sprinkle more comments and assertions. - Cache more stuff in filedesc_t. - Fix numerous minor bugs spotted along the way. - Restructure how the open files array is maintained, for clarity and so that we can eliminate the membar_consumer() call in fd_getfile(). This is mostly syntactic sugar; the main functional change is that fd_nfiles now lives alongside the open file array. Some measurements with libmicro: - simple file syscalls are like close() are between 1 to 10% faster. - some nice improvements, e.g. poll(1000) which is ~50% faster.
2009-05-25 01:41:25 +04:00
&fdp->fd_dt->dt_ff[kn->kn_id]->ff_knlist;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if ((int)kn->kn_id > fdp->fd_lastkqfile)
fdp->fd_lastkqfile = kn->kn_id;
}
SLIST_INSERT_HEAD(list, kn, kn_link);
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
/*
* N.B. kn->kn_fop may change as the result
* of filter_attach()!
*/
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
knote_foplock_enter(kn);
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
error = filter_attach(kn);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (error != 0) {
Put previous removed diagnostic back as debug. It has caught in the past (and now) different kqueue behavior between NetBSD and other kqueue implementations that depend on specific file types. If 3rd party programs trigger this it is probably because we are doing something different.
2017-11-30 17:19:27 +03:00
#ifdef DEBUG
print which process asked for an unsupported event so we can fix it.
2019-10-18 22:43:49 +03:00
struct proc *p = curlwp->l_proc;
add fo_name so we can identify the fileops in a simple way.
2017-11-30 23:25:54 +03:00
const file_t *ft = kn->kn_obj;
print which process asked for an unsupported event so we can fix it.
2019-10-18 22:43:49 +03:00
printf("%s: %s[%d]: event type %d not "
"supported for file type %d/%s "
"(error %d)\n", __func__,
p->p_comm, p->p_pid,
add fo_name so we can identify the fileops in a simple way.
2017-11-30 23:25:54 +03:00
kn->kn_filter, ft ? ft->f_type : -1,
ft ? ft->f_ops->fo_name : "?", error);
Put previous removed diagnostic back as debug. It has caught in the past (and now) different kqueue behavior between NetBSD and other kqueue implementations that depend on specific file types. If 3rd party programs trigger this it is probably because we are doing something different.
2017-11-30 17:19:27 +03:00
#endif
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
fail_ev_add:
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
/*
* N.B. no need to check for this note to
* be in-flux, since it was never visible
* to the monitored object.
*
* knote_detach() drops fdp->fd_lock
*/
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
knote_foplock_exit(kn);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
mutex_enter(&kq->kq_lock);
KNOTE_WILLDETACH(kn);
KASSERT(kn_in_flux(kn) == false);
mutex_exit(&kq->kq_lock);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
knote_detach(kn, fdp, false);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
goto done;
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
atomic_inc_uint(&kfilter->refcnt);
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
goto done_ev_add;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
} else {
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
/* No matching knote and the EV_ADD flag is not set. */
error = ENOENT;
goto doneunlock;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
}
if (kev->flags & EV_DELETE) {
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
/*
* Let the world know that this knote is about to go
* away, and wait for it to settle if it's currently
* in-flux.
*/
mutex_spin_enter(&kq->kq_lock);
if (kn->kn_status & KN_WILLDETACH) {
/*
* This knote is already on its way out,
* so just be done.
*/
mutex_spin_exit(&kq->kq_lock);
goto doneunlock;
}
KNOTE_WILLDETACH(kn);
if (kn_in_flux(kn)) {
mutex_exit(&fdp->fd_lock);
/*
* It's safe for us to conclusively wait for
* this knote to settle because we know we'll
* be completing the detach.
*/
kn_wait_flux(kn, true);
KASSERT(kn_in_flux(kn) == false);
mutex_spin_exit(&kq->kq_lock);
mutex_enter(&fdp->fd_lock);
} else {
mutex_spin_exit(&kq->kq_lock);
}
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
/* knote_detach() drops fdp->fd_lock */
knote_detach(kn, fdp, true);
goto done;
}
/*
* The user may change some filter values after the
* initial EV_ADD, but doing so will not reset any
* filter which have already been triggered.
*/
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
knote_foplock_enter(kn);
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
kn->kn_kevent.udata = kev->udata;
KASSERT(kn->kn_fop != NULL);
Change the kqueue filterops::f_isfd field to filterops::f_flags, and define a flag FILTEROP_ISFD that has the meaning of the prior f_isfd. Field and flag name aligned with OpenBSD. This does not constitute a functional or ABI change, as the field location and size, and the value placed in that field, are the same as the previous code, but we're bumping __NetBSD_Version__ so 3rd-party module source code can adapt, as needed. NetBSD 9.99.89
2021-09-26 04:16:07 +03:00
if (!(kn->kn_fop->f_flags & FILTEROP_ISFD) &&
kn->kn_fop->f_touch != NULL) {
call f_touch with kq_lock held, and without KERNEL_LOCK() - for this adjust EVFILT_USER, which is the only filter actually using that hook kqueue_scan() now doesn't need to exit/enter the kq_lock when calling f_touch, which removes another possible race part of PR kern/50094
2021-01-26 22:09:18 +03:00
mutex_spin_enter(&kq->kq_lock);
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
error = filter_touch(kn, kev, EVENT_REGISTER);
call f_touch with kq_lock held, and without KERNEL_LOCK() - for this adjust EVFILT_USER, which is the only filter actually using that hook kqueue_scan() now doesn't need to exit/enter the kq_lock when calling f_touch, which removes another possible race part of PR kern/50094
2021-01-26 22:09:18 +03:00
mutex_spin_exit(&kq->kq_lock);
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
if (__predict_false(error != 0)) {
/* Never a new knote (which would consume newkn). */
KASSERT(newkn != NULL);
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
knote_foplock_exit(kn);
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
goto doneunlock;
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
} else {
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
kn->kn_sfflags = kev->fflags;
kn->kn_sdata = kev->data;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
/*
* We can get here if we are trying to attach
* an event to a file descriptor that does not
* support events, and the attach routine is
* broken and does not return an error.
*/
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
done_ev_add:
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
rv = filter_event(kn, 0, false);
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
if (rv)
knote_activate(kn);
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
knote_foplock_exit(kn);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/* disable knote */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if ((kev->flags & EV_DISABLE)) {
mutex_spin_enter(&kq->kq_lock);
if ((kn->kn_status & KN_DISABLED) == 0)
kn->kn_status |= KN_DISABLED;
mutex_spin_exit(&kq->kq_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/* enable knote */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if ((kev->flags & EV_ENABLE)) {
knote_enqueue(kn);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
doneunlock:
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_exit(&fdp->fd_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
done:
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_exit(&kqueue_filter_lock);
if (newkn != NULL)
Funnel knote alloc/free into a single pair of functions. NFCI.
2022-07-13 06:23:07 +03:00
knote_free(newkn);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (fp != NULL)
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fd_putfile(fd);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (error);
}
more debug info
2017-09-17 02:55:16 +03:00
#define KN_FMT(buf, kn) \
(snprintb((buf), sizeof(buf), __KN_FLAG_BITS, (kn)->kn_status), buf)
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
#if defined(DDB)
void
kqueue_printit(struct kqueue *kq, bool full, void (*pr)(const char *, ...))
{
const struct knote *kn;
u_int count;
int nmarker;
char buf[128];
count = 0;
nmarker = 0;
(*pr)("kqueue %p (restart=%d count=%u):\n", kq,
!!(kq->kq_count & KQ_RESTART), KQ_COUNT(kq));
(*pr)(" Queued knotes:\n");
TAILQ_FOREACH(kn, &kq->kq_head, kn_tqe) {
if (kn->kn_status & KN_MARKER) {
nmarker++;
} else {
count++;
}
(*pr)(" knote %p: kq=%p status=%s\n",
kn, kn->kn_kq, KN_FMT(buf, kn));
(*pr)(" id=0x%lx (%lu) filter=%d\n",
(u_long)kn->kn_id, (u_long)kn->kn_id, kn->kn_filter);
if (kn->kn_kq != kq) {
(*pr)(" !!! kn->kn_kq != kq\n");
}
}
if (count != KQ_COUNT(kq)) {
(*pr)(" !!! count(%u) != KQ_COUNT(%u)\n",
count, KQ_COUNT(kq));
}
}
#endif /* DDB */
#if defined(DEBUG)
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
static void
more debug info
2017-09-17 02:55:16 +03:00
kqueue_check(const char *func, size_t line, const struct kqueue *kq)
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
{
const struct knote *kn;
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
u_int count;
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
int nmarker;
more debug info
2017-09-17 02:55:16 +03:00
char buf[128];
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
KASSERT(mutex_owned(&kq->kq_lock));
count = 0;
nmarker = 0;
TAILQ_FOREACH(kn, &kq->kq_head, kn_tqe) {
if ((kn->kn_status & (KN_MARKER | KN_QUEUED)) == 0) {
more debug info
2017-09-17 02:55:16 +03:00
panic("%s,%zu: kq=%p kn=%p !(MARKER|QUEUED) %s",
func, line, kq, kn, KN_FMT(buf, kn));
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
}
if ((kn->kn_status & KN_MARKER) == 0) {
if (kn->kn_kq != kq) {
more debug info
2017-09-17 02:55:16 +03:00
panic("%s,%zu: kq=%p kn(%p) != kn->kq(%p): %s",
func, line, kq, kn, kn->kn_kq,
KN_FMT(buf, kn));
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
}
if ((kn->kn_status & KN_ACTIVE) == 0) {
more debug info
2017-09-17 02:55:16 +03:00
panic("%s,%zu: kq=%p kn=%p: !ACTIVE %s",
func, line, kq, kn, KN_FMT(buf, kn));
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
}
count++;
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
if (count > KQ_COUNT(kq)) {
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
panic("%s,%zu: kq=%p kq->kq_count(%u) != "
adjust kq_check() (enabled with DEBUG) to new reality - it's now perfectly normal to have kq_count bigger than number of the linked entries on the kqueue PR kern/50094, problem pointed out by Chuck Silvers
2021-01-21 21:09:23 +03:00
"count(%d), nmarker=%d",
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
func, line, kq, KQ_COUNT(kq), count,
adjust kq_check() (enabled with DEBUG) to new reality - it's now perfectly normal to have kq_count bigger than number of the linked entries on the kqueue PR kern/50094, problem pointed out by Chuck Silvers
2021-01-21 21:09:23 +03:00
nmarker);
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
}
} else {
nmarker++;
}
}
}
more debug info
2017-09-17 02:55:16 +03:00
#define kq_check(a) kqueue_check(__func__, __LINE__, (a))
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
#else /* defined(DEBUG) */
#define kq_check(a) /* nothing */
#endif /* defined(DEBUG) */
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
static void
kqueue_restart(file_t *fp)
{
struct kqueue *kq = fp->f_kqueue;
KASSERT(kq != NULL);
mutex_spin_enter(&kq->kq_lock);
kq->kq_count |= KQ_RESTART;
cv_broadcast(&kq->kq_cv);
mutex_spin_exit(&kq->kq_lock);
}
file(9): New fo_fpathconf operation. XXX kernel revbump -- struct fileops API and ABI change
2023-04-22 16:52:54 +03:00
static int
kqueue_fpathconf(struct file *fp, int name, register_t *retval)
{
return EINVAL;
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
* Scan through the list of events on fp (for a maximum of maxevents),
* returning the results in to ulistp. Timeout is determined by tsp; if
* NULL, wait indefinitely, if 0 valued, perform a poll, otherwise wait
* as appropriate.
*/
static int
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kqueue_scan(file_t *fp, size_t maxevents, struct kevent *ulistp,
const struct timespec *tsp, register_t *retval,
const struct kevent_ops *keops, struct kevent *kevbuf,
size_t kevcnt)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
struct kqueue *kq;
struct kevent *kevp;
Move the internal poll/select related API's to use timespec instead of timeval (rides the uvm bump).
2009-03-29 23:21:19 +04:00
struct timespec ats, sleepts;
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
struct knote *kn, *marker;
struct knote_impl morker;
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
size_t count, nkev, nevents;
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
int timeout, error, touch, rv, influx;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
filedesc_t *fdp;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fdp = curlwp->l_fd;
Try not to use f_data, use f_{vnode,socket,pipe,mqueue,kqueue,ksem} to get a correctly typed pointer.
2014-09-05 13:20:59 +04:00
kq = fp->f_kqueue;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
count = maxevents;
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
nkev = nevents = error = 0;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (count == 0) {
*retval = 0;
return 0;
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
simplify timeout handling code in kqueue_scan()
2003-02-21 23:57:09 +03:00
if (tsp) { /* timeout supplied */
fix erroneously deleted assignment.
2009-03-31 02:22:44 +04:00
ats = *tsp;
Move the internal poll/select related API's to use timespec instead of timeval (rides the uvm bump).
2009-03-29 23:21:19 +04:00
if (inittimeleft(&ats, &sleepts) == -1) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
*retval = maxevents;
return EINVAL;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
Move the internal poll/select related API's to use timespec instead of timeval (rides the uvm bump).
2009-03-29 23:21:19 +04:00
timeout = tstohz(&ats);
simplify timeout handling code in kqueue_scan()
2003-02-21 23:57:09 +03:00
if (timeout <= 0)
reduce sleep time by slept time for retrys
2006-07-15 02:35:15 +04:00
timeout = -1; /* do poll */
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
} else {
simplify timeout handling code in kqueue_scan()
2003-02-21 23:57:09 +03:00
/* no timeout, wait forever */
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
timeout = 0;
Nix trailing whitespace. No functional change.
2017-07-03 03:53:33 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
PR/50730: Benny Siegert: Go kqueue test panics kernel. - use a marker knote from the stack instead of allocating and freeing on each scan. - add more KASSERTS - introduce a KN_BUSY bit that indicates that the knote is currently being scanned, so that knote_detach does not end up deleting it when the file descriptor gets closed and we don't end up using/trashing free memory from the scan.
2016-01-31 07:40:01 +03:00
memset(&morker, 0, sizeof(morker));
Make some knote implementation details private to kern_event.c. NFC, and no ABI change for kevent providers.
2022-07-19 03:46:00 +03:00
marker = &morker.ki_knote;
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
marker->kn_kq = kq;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
marker->kn_status = KN_MARKER;
mutex_spin_enter(&kq->kq_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
retry:
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kevp = kevbuf;
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
if (KQ_COUNT(kq) == 0) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (timeout >= 0) {
error = cv_timedwait_sig(&kq->kq_cv,
&kq->kq_lock, timeout);
if (error == 0) {
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
if (KQ_COUNT(kq) == 0 &&
(kq->kq_count & KQ_RESTART)) {
/* return to clear file reference */
error = ERESTART;
} else if (tsp == NULL || (timeout =
gettimeleft(&ats, &sleepts)) > 0) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
goto retry;
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
} else {
/* don't restart after signals... */
if (error == ERESTART)
error = EINTR;
if (error == EWOULDBLOCK)
error = 0;
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
fix file descriptor locking (from joerg). fixes kernel crashes by running go XXX: pullup-7
2017-07-01 23:08:56 +03:00
mutex_spin_exit(&kq->kq_lock);
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
goto done;
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
/* mark end of knote list */
TAILQ_INSERT_TAIL(&kq->kq_head, marker, kn_tqe);
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
influx = 0;
fix file descriptor locking (from joerg). fixes kernel crashes by running go XXX: pullup-7
2017-07-01 23:08:56 +03:00
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
/*
* Acquire the fdp->fd_lock interlock to avoid races with
* file creation/destruction from other threads.
*/
mutex_spin_exit(&kq->kq_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
relock:
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
mutex_enter(&fdp->fd_lock);
mutex_spin_enter(&kq->kq_lock);
while (count != 0) {
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
/*
* Get next knote. We are guaranteed this will never
* be NULL because of the marker we inserted above.
*/
kn = TAILQ_FIRST(&kq->kq_head);
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
bool kn_is_other_marker =
(kn->kn_status & KN_MARKER) != 0 && kn != marker;
bool kn_is_detaching = (kn->kn_status & KN_WILLDETACH) != 0;
bool kn_is_in_flux = kn_in_flux(kn);
/*
* If we found a marker that's not ours, or this knote
* is in a state of flux, then wait for everything to
* settle down and go around again.
*/
if (kn_is_other_marker || kn_is_detaching || kn_is_in_flux) {
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
if (influx) {
influx = 0;
KQ_FLUX_WAKEUP(kq);
}
mutex_exit(&fdp->fd_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
if (kn_is_other_marker || kn_is_in_flux) {
KQ_FLUX_WAIT(kq);
mutex_spin_exit(&kq->kq_lock);
} else {
/*
* Detaching but not in-flux? Someone is
* actively trying to finish the job; just
* go around and try again.
*/
KASSERT(kn_is_detaching);
mutex_spin_exit(&kq->kq_lock);
preempt_point();
}
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
goto relock;
}
TAILQ_REMOVE(&kq->kq_head, kn, kn_tqe);
if (kn == marker) {
/* it's our marker, stop */
KQ_FLUX_WAKEUP(kq);
if (count == maxevents) {
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
mutex_exit(&fdp->fd_lock);
goto retry;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
break;
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
}
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
KASSERT((kn->kn_status & KN_BUSY) == 0);
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
kq_check(kq);
put back clearing of KN_QUEUED and check for re-queue - as rev. 1.53 notes, it's necessary for correct function fixes PR kern/55946, thanks to Paul Goyette for testing part of PR kern/50094 fix
2021-01-25 22:57:05 +03:00
kn->kn_status &= ~KN_QUEUED;
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
kn->kn_status |= KN_BUSY;
kq_check(kq);
if (kn->kn_status & KN_DISABLED) {
put back clearing of KN_QUEUED and check for re-queue - as rev. 1.53 notes, it's necessary for correct function fixes PR kern/55946, thanks to Paul Goyette for testing part of PR kern/50094 fix
2021-01-25 22:57:05 +03:00
kn->kn_status &= ~KN_BUSY;
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
kq->kq_count--;
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
/* don't want disabled events */
continue;
}
if ((kn->kn_flags & EV_ONESHOT) == 0) {
mutex_spin_exit(&kq->kq_lock);
KASSERT(mutex_owned(&fdp->fd_lock));
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
knote_foplock_enter(kn);
rv = filter_event(kn, 0, false);
knote_foplock_exit(kn);
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
mutex_spin_enter(&kq->kq_lock);
put back clearing of KN_QUEUED and check for re-queue - as rev. 1.53 notes, it's necessary for correct function fixes PR kern/55946, thanks to Paul Goyette for testing part of PR kern/50094 fix
2021-01-25 22:57:05 +03:00
/* Re-poll if note was re-enqueued. */
if ((kn->kn_status & KN_QUEUED) != 0) {
kn->kn_status &= ~KN_BUSY;
/* Re-enqueue raised kq_count, lower it again */
kq->kq_count--;
influx = 1;
continue;
}
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
if (rv == 0) {
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
/*
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
* non-ONESHOT event that hasn't triggered
* again, so it will remain de-queued.
PR/55663: Ruslan Nikolaev: Add support for EVFILT_USER in kqueue(2)
2020-10-31 04:08:31 +03:00
*/
put back clearing of KN_QUEUED and check for re-queue - as rev. 1.53 notes, it's necessary for correct function fixes PR kern/55946, thanks to Paul Goyette for testing part of PR kern/50094 fix
2021-01-25 22:57:05 +03:00
kn->kn_status &= ~(KN_ACTIVE|KN_BUSY);
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
kq->kq_count--;
influx = 1;
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
continue;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
} else {
/*
Fix a regression introduced in kern_event.c,v 1.129 that would cause "udata" to get clobbered on ONESHOT events, and add a unit test for it. Reported by martin@ (manifested in his case as a KASSERT() firing when running unit tests in COMPAT_NETBSD32).
2021-10-23 21:46:26 +03:00
* Must NOT drop kq_lock until we can do
* the KNOTE_WILLDETACH() below.
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
*/
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
}
KASSERT(kn->kn_fop != NULL);
Change the kqueue filterops::f_isfd field to filterops::f_flags, and define a flag FILTEROP_ISFD that has the meaning of the prior f_isfd. Field and flag name aligned with OpenBSD. This does not constitute a functional or ABI change, as the field location and size, and the value placed in that field, are the same as the previous code, but we're bumping __NetBSD_Version__ so 3rd-party module source code can adapt, as needed. NetBSD 9.99.89
2021-09-26 04:16:07 +03:00
touch = (!(kn->kn_fop->f_flags & FILTEROP_ISFD) &&
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
kn->kn_fop->f_touch != NULL);
/* XXXAD should be got from f_event if !oneshot. */
Fix a regression introduced in kern_event.c,v 1.129 that would cause "udata" to get clobbered on ONESHOT events, and add a unit test for it. Reported by martin@ (manifested in his case as a KASSERT() firing when running unit tests in COMPAT_NETBSD32).
2021-10-23 21:46:26 +03:00
KASSERT((kn->kn_status & KN_WILLDETACH) == 0);
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
if (touch) {
Allow the f_touch() filter op to return an error, specifically in the EVENT_REGISTER case.
2021-10-21 05:34:03 +03:00
(void)filter_touch(kn, kevp, EVENT_PROCESS);
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
} else {
*kevp = kn->kn_kevent;
}
kevp++;
nkev++;
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
influx = 1;
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
if (kn->kn_flags & EV_ONESHOT) {
/* delete ONESHOT events after retrieval */
Fix a regression introduced in kern_event.c,v 1.129 that would cause "udata" to get clobbered on ONESHOT events, and add a unit test for it. Reported by martin@ (manifested in his case as a KASSERT() firing when running unit tests in COMPAT_NETBSD32).
2021-10-23 21:46:26 +03:00
KNOTE_WILLDETACH(kn);
put back clearing of KN_QUEUED and check for re-queue - as rev. 1.53 notes, it's necessary for correct function fixes PR kern/55946, thanks to Paul Goyette for testing part of PR kern/50094 fix
2021-01-25 22:57:05 +03:00
kn->kn_status &= ~KN_BUSY;
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
kq->kq_count--;
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
KASSERT(kn_in_flux(kn) == false);
kern: KASSERT(A && B) -> KASSERT(A); KASSERT(B)
2023-04-09 12:18:09 +03:00
KASSERT((kn->kn_status & KN_WILLDETACH) != 0);
KASSERT(kn->kn_kevent.udata == curlwp);
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
mutex_spin_exit(&kq->kq_lock);
knote_detach(kn, fdp, true);
mutex_enter(&fdp->fd_lock);
mutex_spin_enter(&kq->kq_lock);
} else if (kn->kn_flags & EV_CLEAR) {
/* clear state after retrieval */
kn->kn_data = 0;
kn->kn_fflags = 0;
/*
* Manually clear knotes who weren't
* 'touch'ed.
*/
if (touch == 0) {
kn->kn_data = 0;
kn->kn_fflags = 0;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
put back clearing of KN_QUEUED and check for re-queue - as rev. 1.53 notes, it's necessary for correct function fixes PR kern/55946, thanks to Paul Goyette for testing part of PR kern/50094 fix
2021-01-25 22:57:05 +03:00
kn->kn_status &= ~(KN_ACTIVE|KN_BUSY);
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
kq->kq_count--;
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
} else if (kn->kn_flags & EV_DISPATCH) {
kn->kn_status |= KN_DISABLED;
put back clearing of KN_QUEUED and check for re-queue - as rev. 1.53 notes, it's necessary for correct function fixes PR kern/55946, thanks to Paul Goyette for testing part of PR kern/50094 fix
2021-01-25 22:57:05 +03:00
kn->kn_status &= ~(KN_ACTIVE|KN_BUSY);
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
kq->kq_count--;
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
} else {
/* add event back on list */
kq_check(kq);
put back clearing of KN_QUEUED and check for re-queue - as rev. 1.53 notes, it's necessary for correct function fixes PR kern/55946, thanks to Paul Goyette for testing part of PR kern/50094 fix
2021-01-25 22:57:05 +03:00
kn->kn_status |= KN_QUEUED;
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
kn->kn_status &= ~KN_BUSY;
TAILQ_INSERT_TAIL(&kq->kq_head, kn, kn_tqe);
kq_check(kq);
}
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
if (nkev == kevcnt) {
/* do copyouts in kevcnt chunks */
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
influx = 0;
KQ_FLUX_WAKEUP(kq);
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
mutex_spin_exit(&kq->kq_lock);
mutex_exit(&fdp->fd_lock);
error = (*keops->keo_put_events)
(keops->keo_private,
kevbuf, ulistp, nevents, nkev);
mutex_enter(&fdp->fd_lock);
mutex_spin_enter(&kq->kq_lock);
nevents += nkev;
nkev = 0;
kevp = kevbuf;
}
count--;
if (error != 0 || count == 0) {
/* remove marker */
TAILQ_REMOVE(&kq->kq_head, marker, kn_tqe);
break;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
fix file descriptor locking (from joerg). fixes kernel crashes by running go XXX: pullup-7
2017-07-01 23:08:56 +03:00
}
fix a race in kqueue_scan() - when multiple threads check the same kqueue, it could happen other thread seen empty kqueue while kevent was being checked for re-firing and re-queued make sure to keep retrying if there are outstanding kevents even if no kevent is found on first pass through the queue, and only drop the KN_QUEUED flag and kq_count when actually completely done with the kevent change is inspired by the FreeBSD in-flux handling, but without introducing the reference counting PR kern/50094 by Christof Meerwald
2021-01-21 00:39:09 +03:00
KQ_FLUX_WAKEUP(kq);
reduce indentation for the main processing loop in kqueue_scan(), this also makes the code more similar to FreeBSD; NFCI part of PR kern/50094
2020-12-27 15:45:33 +03:00
mutex_spin_exit(&kq->kq_lock);
mutex_exit(&fdp->fd_lock);
done:
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (nkev != 0) {
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/* copyout remaining events */
- Split sys_kevent into kevent1 so that it can be used by COMPAT_NETBSD32 code. - To achieve COMPAT_NETBSD32 compatibility, introduce a parameter to kevent1 that points to functions that do the actual copyin/copyout operations. This is similar to what was done in FreeBSD by Paul Saab. - Add the COMPAT_NETBSD32 definitions and hooks.
2005-10-23 05:33:32 +04:00
error = (*keops->keo_put_events)(keops->keo_private,
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kevbuf, ulistp, nevents, nkev);
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*retval = maxevents - count;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return error;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* fileops ioctl method for a kqueue descriptor.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*
* Two ioctls are currently supported. They both use struct kfilter_mapping:
* KFILTER_BYNAME find name for filter, and return result in
* name, which is of size len.
* KFILTER_BYFILTER find filter for name. len is ignored.
*/
/*ARGSUSED*/
static int
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kqueue_ioctl(file_t *fp, u_long com, void *data)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
struct kfilter_mapping *km;
const struct kfilter *kfilter;
char *name;
int error;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
km = data;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
error = 0;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
name = kmem_alloc(KFILTER_MAXNAME, KM_SLEEP);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
switch (com) {
case KFILTER_BYFILTER: /* convert filter -> name */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_enter(&kqueue_filter_lock, RW_READER);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kfilter = kfilter_byfilter(km->filter);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (kfilter != NULL) {
strlcpy(name, kfilter->name, KFILTER_MAXNAME);
rw_exit(&kqueue_filter_lock);
error = copyoutstr(name, km->name, km->len, NULL);
} else {
rw_exit(&kqueue_filter_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
error = ENOENT;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
break;
case KFILTER_BYNAME: /* convert name -> filter */
error = copyinstr(km->name, name, KFILTER_MAXNAME, NULL);
if (error) {
break;
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_enter(&kqueue_filter_lock, RW_READER);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kfilter = kfilter_byname(name);
if (kfilter != NULL)
km->filter = kfilter->filter;
else
error = ENOENT;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
rw_exit(&kqueue_filter_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
break;
default:
error = ENOTTY;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
break;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kmem_free(name, KFILTER_MAXNAME);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (error);
}
/*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* fileops fcntl method for a kqueue descriptor.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
static int
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kqueue_fcntl(file_t *fp, u_int com, void *data)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
return (ENOTTY);
}
/*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* fileops poll method for a kqueue descriptor.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
* Determine if kqueue has events pending.
*/
static int
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kqueue_poll(file_t *fp, int events)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
struct kqueue *kq;
int revents;
Try not to use f_data, use f_{vnode,socket,pipe,mqueue,kqueue,ksem} to get a correctly typed pointer.
2014-09-05 13:20:59 +04:00
kq = fp->f_kqueue;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
revents = 0;
if (events & (POLLIN | POLLRDNORM)) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_spin_enter(&kq->kq_lock);
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
if (KQ_COUNT(kq) != 0) {
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
revents |= events & (POLLIN | POLLRDNORM);
} else {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
selrecord(curlwp, &kq->kq_sel);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
kq_check(kq);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_spin_exit(&kq->kq_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return revents;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* fileops stat method for a kqueue descriptor.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
* Returns dummy info, with st_size being number of events pending.
*/
static int
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kqueue_stat(file_t *fp, struct stat *st)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct kqueue *kq;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Try not to use f_data, use f_{vnode,socket,pipe,mqueue,kqueue,ksem} to get a correctly typed pointer.
2014-09-05 13:20:59 +04:00
kq = fp->f_kqueue;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
memset(st, 0, sizeof(*st));
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
st->st_size = KQ_COUNT(kq);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
st->st_blksize = sizeof(struct kevent);
Make the info returned by kqueue_stat() a little less barren.
2021-09-30 04:20:53 +03:00
st->st_mode = S_IFIFO | S_IRUSR | S_IWUSR;
st->st_blocks = 1;
st->st_uid = kauth_cred_geteuid(fp->f_cred);
st->st_gid = kauth_cred_getegid(fp->f_cred);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
return 0;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
static void
kqueue_doclose(struct kqueue *kq, struct klist *list, int fd)
{
struct knote *kn;
filedesc_t *fdp;
fdp = kq->kq_fdp;
KASSERT(mutex_owned(&fdp->fd_lock));
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
again:
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
for (kn = SLIST_FIRST(list); kn != NULL;) {
if (kq != kn->kn_kq) {
kn = SLIST_NEXT(kn, kn_link);
continue;
}
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
if (knote_detach_quiesce(kn)) {
mutex_enter(&fdp->fd_lock);
goto again;
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
knote_detach(kn, fdp, true);
mutex_enter(&fdp->fd_lock);
kn = SLIST_FIRST(list);
}
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
/*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* fileops close method for a kqueue descriptor.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
static int
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kqueue_close(file_t *fp)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct kqueue *kq;
filedesc_t *fdp;
fdfile_t *ff;
int i;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Try not to use f_data, use f_{vnode,socket,pipe,mqueue,kqueue,ksem} to get a correctly typed pointer.
2014-09-05 13:20:59 +04:00
kq = fp->f_kqueue;
fp->f_kqueue = NULL;
- initialize kn_id - in close, invalidate f_data and f_type early to prevent accidental re-use - add a DIAGNOSTIC for when we use unsupported fd's and a KASSERT for f_event being NULL.
2012-11-24 19:14:32 +04:00
fp->f_type = 0;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fdp = curlwp->l_fd;
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
KASSERT(kq->kq_fdp == fdp);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_enter(&fdp->fd_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
/*
* We're doing to drop the fd_lock multiple times while
* we detach knotes. During this time, attempts to register
* knotes via the back door (e.g. knote_proc_fork_track())
* need to fail, lest they sneak in to attach a knote after
* we've already drained the list it's destined for.
*
s/aquire/acquire/ in comment.
2022-01-01 13:54:21 +03:00
* We must acquire kq_lock here to set KQ_CLOSING (to serialize
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
* with other code paths that modify kq_count without holding
* the fd_lock), but once this bit is set, it's only safe to
* test it while holding the fd_lock, and holding kq_lock while
* doing so is not necessary.
*/
mutex_enter(&kq->kq_lock);
kq->kq_count |= KQ_CLOSING;
mutex_exit(&kq->kq_lock);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
for (i = 0; i <= fdp->fd_lastkqfile; i++) {
More changes to improve kern_descrip.c. - Avoid atomics in more places. - Remove the per-descriptor mutex, and just use filedesc_t::fd_lock. It was only being used to synchronize close, and in any case we needed to take fd_lock to free the descriptor slot. - Optimize certain paths for the <NDFDFILE case. - Sprinkle more comments and assertions. - Cache more stuff in filedesc_t. - Fix numerous minor bugs spotted along the way. - Restructure how the open files array is maintained, for clarity and so that we can eliminate the membar_consumer() call in fd_getfile(). This is mostly syntactic sugar; the main functional change is that fd_nfiles now lives alongside the open file array. Some measurements with libmicro: - simple file syscalls are like close() are between 1 to 10% faster. - some nice improvements, e.g. poll(1000) which is ~50% faster.
2009-05-25 01:41:25 +04:00
if ((ff = fdp->fd_dt->dt_ff[i]) == NULL)
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
continue;
kqueue_doclose(kq, (struct klist *)&ff->ff_knlist, i);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
if (fdp->fd_knhashmask != 0) {
for (i = 0; i < fdp->fd_knhashmask + 1; i++) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kqueue_doclose(kq, &fdp->fd_knhash[i], -1);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
}
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_exit(&fdp->fd_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
#if defined(DEBUG)
mutex_enter(&kq->kq_lock);
kq_check(kq);
mutex_exit(&kq->kq_lock);
#endif /* DEBUG */
KASSERT(TAILQ_EMPTY(&kq->kq_head));
implement fo_restart hook for kqueue descriptors, so that close(2) on the descriptor won't block indefinitely if other thread is currently blocked on the same kqueue in kevent(2) done similarily to pipes and sockets, i.e. using flag on the potentially shared kqueue structure hooked off file_t - this is somewhat suboptimal if the application dup(2)ped the descriptor, but this should be rare enough to not really matter usually this causes the kevent(2) to end up returning EBADF since on the syscall restart the descriptor is not there anymore; if dup(2)ped the kevent(2) call can continue successfully if the closed kqueue descriptor was other than the one used for the kevent(2) call PR kern/46248 by Julian Fagir
2021-05-02 22:13:43 +03:00
KASSERT(KQ_COUNT(kq) == 0);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_destroy(&kq->kq_lock);
cv_destroy(&kq->kq_cv);
Welcome to 4.99.55: - Add a lot of missing selinit() and seldestroy() calls. - Merge selwakeup() and selnotify() calls into a single selnotify(). - Add an additional 'events' argument to selnotify() call. It will indicate which event (POLL_IN, POLL_OUT, etc) happen. If unknown, zero may be used. Note: please pass appropriate value of 'events' where possible. Proposed on: <tech-kern>
2008-03-01 17:16:49 +03:00
seldestroy(&kq->kq_sel);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kmem_free(kq, sizeof(*kq));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
return (0);
}
/*
* struct fileops kqfilter method for a kqueue descriptor.
* Event triggered when monitored kqueue changes.
*/
static int
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kqueue_kqfilter(file_t *fp, struct knote *kn)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
struct kqueue *kq;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
Try not to use f_data, use f_{vnode,socket,pipe,mqueue,kqueue,ksem} to get a correctly typed pointer.
2014-09-05 13:20:59 +04:00
kq = ((file_t *)kn->kn_obj)->f_kqueue;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
KASSERT(fp == kn->kn_obj);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
if (kn->kn_filter != EVFILT_READ)
In kqueue_kqfilter(), return EINVAL instead of 1 if something other than EVFILT_READ was requested.
2021-09-27 02:37:40 +03:00
return EINVAL;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kn->kn_fop = &kqread_filtops;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_enter(&kq->kq_lock);
Use sel{record,remove}_knote().
2020-12-11 06:00:09 +03:00
selrecord_knote(&kq->kq_sel, kn);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_exit(&kq->kq_lock);
return 0;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* Walk down a list of knotes, activating them if their event has
* triggered. The caller's object lock (e.g. device driver lock)
* must be held.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
void
knote(struct klist *list, long hint)
{
make list traversing in knote() safe against removal of the entry while the loop body is executed -- at least in the EVFILT_PROC / exit case a race condition exists which can cause this fixes a panic triggered eg by tests/kernel/kqueue/proc1
2010-09-10 14:23:46 +04:00
struct knote *kn, *tmpkn;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
make list traversing in knote() safe against removal of the entry while the loop body is executed -- at least in the EVFILT_PROC / exit case a race condition exists which can cause this fixes a panic triggered eg by tests/kernel/kqueue/proc1
2010-09-10 14:23:46 +04:00
SLIST_FOREACH_SAFE(kn, list, kn_selnext, tmpkn) {
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
/*
* We assume here that the backing object's lock is
* already held if we're traversing the klist, and
* so acquiring the knote foplock would create a
* deadlock scenario. But we also know that the klist
* won't disappear on us while we're here, so not
* acquiring it is safe.
*/
if (filter_event(kn, hint, true)) {
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
knote_activate(kn);
In knote(), don't call kn->kn_fop->f_event() directly; use filter_event() to get the correct KERNEL_LOCK handling for the filter attached to that specific note.
2021-09-30 04:12:06 +03:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* Remove all knotes referencing a specified fd
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
void
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
knote_fdclose(int fd)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct klist *list;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
struct knote *kn;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
filedesc_t *fdp;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
again:
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fdp = curlwp->l_fd;
mutex_enter(&fdp->fd_lock);
Load struct filedesc::fd_dt with atomic_load_consume. Exceptions: when fd_refcnt <= 1, or when holding fd_lock. While here: - Restore KASSERT(mutex_owned(&fdp->fd_lock)) in fd_unused. => This is used only in fd_close and fd_abort, where it holds. - Move bounds check assertion in fd_putfile to where it matters. - Store fd_dt with atomic_store_release. - Move load of fd_dt under lock in knote_fdclose. - Omit membar_consumer in fdesc_readdir. => atomic_load_consume serves the same purpose now. => Was needed only on alpha anyway.
2020-02-01 05:23:03 +03:00
list = (struct klist *)&fdp->fd_dt->dt_ff[fd]->ff_knlist;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
while ((kn = SLIST_FIRST(list)) != NULL) {
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
if (knote_detach_quiesce(kn)) {
goto again;
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
knote_detach(kn, fdp, true);
mutex_enter(&fdp->fd_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_exit(&fdp->fd_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
* Drop knote. Called with fdp->fd_lock held, and will drop before
* returning.
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
*/
static void
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
knote_detach(struct knote *kn, filedesc_t *fdp, bool dofop)
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct klist *list;
- kqueue_scan: work around problem noted by yamt@: if an event fires while we have unlocked the kqueue to check its state, leave it queued and re-check later. - knote_dequeue: fold into knote_detach since nothing else uses it. - Note a couple more problems.
2008-03-26 16:32:32 +03:00
struct kqueue *kq;
kq = kn->kn_kq;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
KASSERT((kn->kn_status & KN_MARKER) == 0);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
KASSERT((kn->kn_status & KN_WILLDETACH) != 0);
KASSERT(kn->kn_fop != NULL);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
KASSERT(mutex_owned(&fdp->fd_lock));
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
- kqueue_scan: work around problem noted by yamt@: if an event fires while we have unlocked the kqueue to check its state, leave it queued and re-check later. - knote_dequeue: fold into knote_detach since nothing else uses it. - Note a couple more problems.
2008-03-26 16:32:32 +03:00
/* Remove from monitored object. */
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if (dofop) {
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
knote_foplock_enter(kn);
- Define a new filterops flag FILTEROP_MPSAFE, which states that the kqueue filter does not require the KERNEL_LOCK to be held. - Add wrappers around the calls into the filterops that take care of the locking requirements. No functional change, since no filterops yet define FILTEROP_MPSAFE.
2021-09-26 06:12:50 +03:00
filter_detach(kn);
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
knote_foplock_exit(kn);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
}
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
- kqueue_scan: work around problem noted by yamt@: if an event fires while we have unlocked the kqueue to check its state, leave it queued and re-check later. - knote_dequeue: fold into knote_detach since nothing else uses it. - Note a couple more problems.
2008-03-26 16:32:32 +03:00
/* Remove from descriptor table. */
Change the kqueue filterops::f_isfd field to filterops::f_flags, and define a flag FILTEROP_ISFD that has the meaning of the prior f_isfd. Field and flag name aligned with OpenBSD. This does not constitute a functional or ABI change, as the field location and size, and the value placed in that field, are the same as the previous code, but we're bumping __NetBSD_Version__ so 3rd-party module source code can adapt, as needed. NetBSD 9.99.89
2021-09-26 04:16:07 +03:00
if (kn->kn_fop->f_flags & FILTEROP_ISFD)
More changes to improve kern_descrip.c. - Avoid atomics in more places. - Remove the per-descriptor mutex, and just use filedesc_t::fd_lock. It was only being used to synchronize close, and in any case we needed to take fd_lock to free the descriptor slot. - Optimize certain paths for the <NDFDFILE case. - Sprinkle more comments and assertions. - Cache more stuff in filedesc_t. - Fix numerous minor bugs spotted along the way. - Restructure how the open files array is maintained, for clarity and so that we can eliminate the membar_consumer() call in fd_getfile(). This is mostly syntactic sugar; the main functional change is that fd_nfiles now lives alongside the open file array. Some measurements with libmicro: - simple file syscalls are like close() are between 1 to 10% faster. - some nice improvements, e.g. poll(1000) which is ~50% faster.
2009-05-25 01:41:25 +04:00
list = (struct klist *)&fdp->fd_dt->dt_ff[kn->kn_id]->ff_knlist;
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
else
list = &fdp->fd_knhash[KN_HASH(kn->kn_id, fdp->fd_knhashmask)];
SLIST_REMOVE(list, kn, knote, kn_link);
- kqueue_scan: work around problem noted by yamt@: if an event fires while we have unlocked the kqueue to check its state, leave it queued and re-check later. - knote_dequeue: fold into knote_detach since nothing else uses it. - Note a couple more problems.
2008-03-26 16:32:32 +03:00
/* Remove from kqueue. */
PR/50730: Benny Siegert: Go kqueue test panics kernel. - use a marker knote from the stack instead of allocating and freeing on each scan. - add more KASSERTS - introduce a KN_BUSY bit that indicates that the knote is currently being scanned, so that knote_detach does not end up deleting it when the file descriptor gets closed and we don't end up using/trashing free memory from the scan.
2016-01-31 07:40:01 +03:00
again:
- kqueue_scan: work around problem noted by yamt@: if an event fires while we have unlocked the kqueue to check its state, leave it queued and re-check later. - knote_dequeue: fold into knote_detach since nothing else uses it. - Note a couple more problems.
2008-03-26 16:32:32 +03:00
mutex_spin_enter(&kq->kq_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
KASSERT(kn_in_flux(kn) == false);
- kqueue_scan: work around problem noted by yamt@: if an event fires while we have unlocked the kqueue to check its state, leave it queued and re-check later. - knote_dequeue: fold into knote_detach since nothing else uses it. - Note a couple more problems.
2008-03-26 16:32:32 +03:00
if ((kn->kn_status & KN_QUEUED) != 0) {
kq_check(kq);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
KASSERT(KQ_COUNT(kq) != 0);
PR/50730: Benny Siegert: Go kqueue test panics kernel. - use a marker knote from the stack instead of allocating and freeing on each scan. - add more KASSERTS - introduce a KN_BUSY bit that indicates that the knote is currently being scanned, so that knote_detach does not end up deleting it when the file descriptor gets closed and we don't end up using/trashing free memory from the scan.
2016-01-31 07:40:01 +03:00
kq->kq_count--;
- kqueue_scan: work around problem noted by yamt@: if an event fires while we have unlocked the kqueue to check its state, leave it queued and re-check later. - knote_dequeue: fold into knote_detach since nothing else uses it. - Note a couple more problems.
2008-03-26 16:32:32 +03:00
TAILQ_REMOVE(&kq->kq_head, kn, kn_tqe);
kn->kn_status &= ~KN_QUEUED;
kq_check(kq);
PR/50730: Benny Siegert: Go kqueue test panics kernel. - use a marker knote from the stack instead of allocating and freeing on each scan. - add more KASSERTS - introduce a KN_BUSY bit that indicates that the knote is currently being scanned, so that knote_detach does not end up deleting it when the file descriptor gets closed and we don't end up using/trashing free memory from the scan.
2016-01-31 07:40:01 +03:00
} else if (kn->kn_status & KN_BUSY) {
mutex_spin_exit(&kq->kq_lock);
goto again;
- kqueue_scan: work around problem noted by yamt@: if an event fires while we have unlocked the kqueue to check its state, leave it queued and re-check later. - knote_dequeue: fold into knote_detach since nothing else uses it. - Note a couple more problems.
2008-03-26 16:32:32 +03:00
}
mutex_spin_exit(&kq->kq_lock);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_exit(&fdp->fd_lock);
Change the kqueue filterops::f_isfd field to filterops::f_flags, and define a flag FILTEROP_ISFD that has the meaning of the prior f_isfd. Field and flag name aligned with OpenBSD. This does not constitute a functional or ABI change, as the field location and size, and the value placed in that field, are the same as the previous code, but we're bumping __NetBSD_Version__ so 3rd-party module source code can adapt, as needed. NetBSD 9.99.89
2021-09-26 04:16:07 +03:00
if (kn->kn_fop->f_flags & FILTEROP_ISFD)
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
fd_putfile(kn->kn_id);
atomic_dec_uint(&kn->kn_kfilter->refcnt);
Funnel knote alloc/free into a single pair of functions. NFCI.
2022-07-13 06:23:07 +03:00
knote_free(kn);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
/*
* Queue new event for knote.
*/
static void
knote_enqueue(struct knote *kn)
{
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
struct kqueue *kq;
KASSERT((kn->kn_status & KN_MARKER) == 0);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
kq = kn->kn_kq;
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_spin_enter(&kq->kq_lock);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
if (__predict_false(kn->kn_status & KN_WILLDETACH)) {
/* Don't bother enqueueing a dying knote. */
goto out;
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
if ((kn->kn_status & KN_DISABLED) != 0) {
kn->kn_status &= ~KN_DISABLED;
}
if ((kn->kn_status & (KN_ACTIVE | KN_QUEUED)) == KN_ACTIVE) {
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
kq_check(kq);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kn->kn_status |= KN_QUEUED;
PR/50730: Benny Siegert: Go kqueue test panics kernel. - use a marker knote from the stack instead of allocating and freeing on each scan. - add more KASSERTS - introduce a KN_BUSY bit that indicates that the knote is currently being scanned, so that knote_detach does not end up deleting it when the file descriptor gets closed and we don't end up using/trashing free memory from the scan.
2016-01-31 07:40:01 +03:00
TAILQ_INSERT_TAIL(&kq->kq_head, kn, kn_tqe);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
KASSERT(KQ_COUNT(kq) < KQ_MAXCOUNT);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kq->kq_count++;
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
kq_check(kq);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
cv_broadcast(&kq->kq_cv);
selnotify(&kq->kq_sel, 0, NOTE_SUBMIT);
}
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
out:
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_spin_exit(&kq->kq_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
/*
* Queue new event for knote.
*/
static void
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
knote_activate_locked(struct knote *kn)
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
{
struct kqueue *kq;
KASSERT((kn->kn_status & KN_MARKER) == 0);
kq = kn->kn_kq;
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
if (__predict_false(kn->kn_status & KN_WILLDETACH)) {
/* Don't bother enqueueing a dying knote. */
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
return;
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
}
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kn->kn_status |= KN_ACTIVE;
if ((kn->kn_status & (KN_QUEUED | KN_DISABLED)) == 0) {
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
kq_check(kq);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kn->kn_status |= KN_QUEUED;
PR/50730: Benny Siegert: Go kqueue test panics kernel. - use a marker knote from the stack instead of allocating and freeing on each scan. - add more KASSERTS - introduce a KN_BUSY bit that indicates that the knote is currently being scanned, so that knote_detach does not end up deleting it when the file descriptor gets closed and we don't end up using/trashing free memory from the scan.
2016-01-31 07:40:01 +03:00
TAILQ_INSERT_TAIL(&kq->kq_head, kn, kn_tqe);
Changes to make EVFILT_PROC MP-safe: Because the locking protocol around processes is somewhat complex compared to other events that can be posted on kqueues, introduce new functions for posting NOTE_EXEC, NOTE_EXIT, and NOTE_FORK, rather than just using the generic knote() function. These functions KASSERT() their locking expectations, and deal with other complexities for each situation. knote_proc_fork(), in particiular, needs to handle NOTE_TRACK, which requires allocation of a new knote to attach to the child process. We don't want to be allocating memory while holding the parent's p_lock. Furthermore, we also have to attach the tracking note to the child process, which means we have to acquire the child's p_lock. So, to handle all this, we introduce some additional synchronization infrastructure around the 'knote' structure: - Add the ability to mark a knote as being in a state of flux. Knotes in this state are guaranteed not to be detached/deleted, thus allowing a code path drop other locks after putting a knote in this state. - Code paths that wish to detach/delete a knote must first check if the knote is in-flux. If so, they must wait for it to quiesce. Because multiple threads of execution may attempt this concurrently, a mechanism exists for a single LWP to claim the detach responsibility; all other threads simply wait for the knote to disappear before they can make further progress. - When kqueue_scan() encounters an in-flux knote, it simply treats the situation just like encountering another thread's queue marker -- wait for the flux to settle and continue on. (The "in-flux knote" idea was inspired by FreeBSD, but this works differently from their implementation, as the two kqueue implementations have diverged quite a bit.) knote_proc_fork() uses this infrastructure to implement NOTE_TRACK like so: - Attempt to put the original tracking knote into a state of flux; if this fails (because the note has a detach pending), we skip all processing (the original process has lost interest, and we simply won the race). - Once the note is in-flux, drop the kq and forking process's locks, and allocate 2 knotes: one to post the NOTE_CHILD event, and one to attach a new NOTE_TRACK to the child process. Notably, we do NOT go through kqueue_register() to do this, but rather do all of the work directly and KASSERT() our assumptions; this allows us to directly control our interaction with locks. All memory allocations here are performed with KM_NOSLEEP, in order to prevent holding the original knote in-flux indefinitely. - Because the NOTE_TRACK use case adds knotes to kqueues through a sort of back-door mechanism, we must serialize with the closing of the destination kqueue's file descriptor, so steal another bit from the kq_count field to notify other threads that a kqueue is on its way out to prevent new knotes from being enqueued while the close path detaches them. In addition to fixing EVFILT_PROC's reliance on KERNEL_LOCK, this also fixes a long-standing bug whereby a NOTE_CHILD event could be dropped if the child process exited before the interested process received the NOTE_CHILD event (the same knote would be used to deliver the NOTE_EXIT event, and would clobber the NOTE_CHILD's 'data' field). Add a bunch of comments to explain what's going on in various critical sections, and sprinkle additional KASSERT()s to validate assumptions in several more locations.
2021-10-10 21:07:51 +03:00
KASSERT(KQ_COUNT(kq) < KQ_MAXCOUNT);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
kq->kq_count++;
add some DEBUG checks.
2008-03-24 12:09:55 +03:00
kq_check(kq);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
cv_broadcast(&kq->kq_cv);
selnotify(&kq->kq_sel, 0, NOTE_SUBMIT);
}
- Don't use a separate kqueue_timer_lock; just protect those knotes with the kq->kq_lock. - Re-factor the guts of knote_activate() into knote_activate_locked(), and use it in a few places to avoid a few unlock-the-immediately-lock cycles. - Define a FILT_TIMER_NOSCHED macro, rather than hard-coding (uintptr_t)-1 in a bunch of difference place. NFC.
2021-10-21 03:54:15 +03:00
}
static void
knote_activate(struct knote *kn)
{
struct kqueue *kq = kn->kn_kq;
mutex_spin_enter(&kq->kq_lock);
knote_activate_locked(kn);
File descriptor changes, discussed on tech-kern: - Redo reference counting to be sane. LWPs accessing files take a short term reference on the local file descriptor. This is the most common case. While a file is in a process descriptor table, a reference is held to the file. The file reference count only changes during control operations like open() or close(). Code that comes at files from an unusual direction (i.e. foreign to the process) like procfs or sysctl takes a reference on the file (f_count), and not on a descriptor. - Remove knowledge of reference counting and locking from most code that deals with files. - Make the usual case of file descriptor lookup lockless. - Make kqueue MP and MT safe. PR kern/38098, PR kern/38137. - Fix numerous file handling bugs, and bugs in the descriptor code that affected multithreaded processes. - Split descriptor system calls out into sys_descrip.c. - A few stylistic changes: KNF, remove unused casts now that caddr_t is gone. Replace dumb gotos with loop control in a few places. - Don't do redundant pointer passing (struct proc, lwp, filedesc *) unless the routine is likely to be inlined. Most of the time it's about the current process.
2008-03-22 00:53:35 +03:00
mutex_spin_exit(&kq->kq_lock);
merge kqueue branch into -current kqueue provides a stateful and efficient event notification framework currently supported events include socket, file, directory, fifo, pipe, tty and device changes, and monitoring of processes and signals kqueue is supported by all writable filesystems in NetBSD tree (with exception of Coda) and all device drivers supporting poll(2) based on work done by Jonathan Lemon for FreeBSD initial NetBSD port done by Luke Mewburn and Jason Thorpe
2002-10-23 13:10:23 +04:00
}
Setting EV_EOF requires modifying kn->kn_flags. However, that relies on holding the kq_lock of that note's kq. Rather than exposing this directly, add new knote_set_eof() and knote_clear_eof() functions that handle the necessary locking and don't leak as many implementation details to modules. NetBSD 9.99.91
2021-10-11 04:07:36 +03:00
Support modifying an existing timer without having to delete it first. Semantics match FreeBSD.
2021-10-22 07:49:24 +03:00
static void
knote_deactivate_locked(struct knote *kn)
{
struct kqueue *kq = kn->kn_kq;
if (kn->kn_status & KN_QUEUED) {
kq_check(kq);
kn->kn_status &= ~KN_QUEUED;
TAILQ_REMOVE(&kq->kq_head, kn, kn_tqe);
KASSERT(KQ_COUNT(kq) > 0);
kq->kq_count--;
kq_check(kq);
}
kn->kn_status &= ~KN_ACTIVE;
}
Setting EV_EOF requires modifying kn->kn_flags. However, that relies on holding the kq_lock of that note's kq. Rather than exposing this directly, add new knote_set_eof() and knote_clear_eof() functions that handle the necessary locking and don't leak as many implementation details to modules. NetBSD 9.99.91
2021-10-11 04:07:36 +03:00
/*
* Set EV_EOF on the specified knote. Also allows additional
* EV_* flags to be set (e.g. EV_ONESHOT).
*/
void
knote_set_eof(struct knote *kn, uint32_t flags)
{
struct kqueue *kq = kn->kn_kq;
mutex_spin_enter(&kq->kq_lock);
kn->kn_flags |= EV_EOF | flags;
mutex_spin_exit(&kq->kq_lock);
}
/*
* Clear EV_EOF on the specified knote.
*/
void
knote_clear_eof(struct knote *kn)
{
struct kqueue *kq = kn->kn_kq;
mutex_spin_enter(&kq->kq_lock);
kn->kn_flags &= ~EV_EOF;
mutex_spin_exit(&kq->kq_lock);
}
Move klist_{init,fini,insert,remove}() into kern_event.c. NFC.
2022-07-13 17:11:46 +03:00
/*
* Initialize a klist.
*/
void
klist_init(struct klist *list)
{
SLIST_INIT(list);
}
/*
* Finalize a klist.
*/
void
klist_fini(struct klist *list)
{
Fix a problem whereby detaching a device that has open kevent registrations can result in a UAF: When a device detaches, it calls seldestroy(), which calls knote_fini(), and when that returns, the softc that contained the selinfo and klist are freed. However, any knotes that were registered still linger on with the kq descriptor they're were associated with, and when the file descriptors close, those knotes will be f_detach'd, which will call into the driver instance that no longer exists. Address this problem by adding a "foplock" mutex to the knote. This foplock must be held when calling into filter_attach(), filter_detach(), and filter_event() (XXX not filter_touch(); see code for details). Now, in klist_fini(), for each knote that is on the klist that's about to be blown away, acquire the foplock, replace the knote's filterops with a do-nothing stub, and release the foplock. The end result is that: ==> The foplock ensures that calls into filter_*() will get EITHER the real backing object's filterops OR the nop stubs. ==> Holing the foplock across the filter_*() calls ensures that klist_fini() will not complete until there are no callers inside the filterops that are about to be blown away.
2022-07-19 04:03:05 +03:00
struct knote *kn;
/*
* Neuter all existing knotes on the klist because the list is
* being destroyed. The caller has guaranteed that no additional
* knotes will be added to the list, that the backing object's
* locks are not held (otherwise there is a locking order issue
* with acquiring the knote foplock ), and that we can traverse
* the list safely in this state.
*/
SLIST_FOREACH(kn, list, kn_selnext) {
knote_foplock_enter(kn);
KASSERT(kn->kn_fop != NULL);
if (kn->kn_fop->f_flags & FILTEROP_ISFD) {
kn->kn_fop = &nop_fd_filtops;
} else {
kn->kn_fop = &nop_filtops;
}
knote_foplock_exit(kn);
}
Move klist_{init,fini,insert,remove}() into kern_event.c. NFC.
2022-07-13 17:11:46 +03:00
}
/*
* Insert a knote into a klist.
*/
void
klist_insert(struct klist *list, struct knote *kn)
{
SLIST_INSERT_HEAD(list, kn, kn_selnext);
}
/*
* Remove a knote from a klist. Returns true if the last
* knote was removed and the list is now empty.
*/
bool
klist_remove(struct klist *list, struct knote *kn)
{
SLIST_REMOVE(list, kn, knote, kn_selnext);
return SLIST_EMPTY(list);
}