The suspension helpers are now put into file system specific operations.
This means every file system not supporting these helpers cannot be suspended
and therefore snapshots are no longer possible.
Implemented for file systems of type ffs.
The new API is enabled on a kernel option NEWVNGATE. This option is
not enabled by default in any kernel config.
Presented and discussed on tech-kern with much input from
Bill Studenmund <wrstuden@netbsd.org> and YAMAMOTO Takashi <yamt@netbsd.org>.
Welcome to 4.99.9 (new vfs op vfs_suspendctl).
- Make procfs_control() in procfs_ctl.c static,
- Add an argument to the above, 'pfs', for the pfsnode,
- Add another request type to KAUTH_PROCESS_CANPROCFS named
KAUTH_REQ_PROCESS_CANPROCFS_CTL (and update documentation),
- Use the above combination in a call to kauth_authorize_process().
- LOCKPARENT is no longer relevant for lookup(), relookup() or VOP_LOOKUP().
these now always return the parent vnode locked. namei() works as before.
lookup() and various other paths no longer acquire vnode locks in the
wrong order via vrele(). fixes PR 32535.
as a nice side effect, path lookup is also up to 25% faster.
- the above allows us to get rid of PDIRUNLOCK.
- also get rid of WANTPARENT (just use LOCKPARENT and unlock it).
- remove an assumption in layer_node_find() that all file systems implement
a recursive VOP_LOCK() (unionfs doesn't).
- require that all file systems supply vfs_vptofh and vfs_fhtovp routines.
fill in eopnotsupp() for file systems that don't support being exported
and remove the checks for NULL. (layerfs calls these without checking.)
- in union_lookup1(), don't change refcounts in the ISDOTDOT case, just
adjust which vnode is locked. fixes PR 33374.
- apply fixes for ufs_rename() from ufs_vnops.c rev. 1.61 to ext2fs_rename().
unlocked instead of locked. chuq agrees
* use slock set to &uobj->vmobjlock also for the last simple lock
operation to be consistent with the rest of the function
First, remove process_checkioperm() calls from MD code. Similar checks
using kauth(9) routines (on the process scope, using appropriate action)
are done in the callers.
Add secmodel back-end to handle each subsystem.
using both the rawio_spec and passthru actions to detect raw disk
activity. Same for kernel memory policy.
Update documentation (no longer need to expose veriexec_rawchk()) and
remove all Veriexec-related bits from specfs.
vp->v_interlock may be unlocked twice: Once explicitly and a second time
implicilty by lockmgr. LK_INTERLOCK is cleared from the variable flags but
not from ap->a_flags which is used with lockmgr. This is not so much of a
problem because there seems to be no call site that actually uses
LK_INTERLOCK with layer_unlock or VOP_UNLOCK.
okay martin@
vnodes were synced and processed backwards. This meant that the last
accessed node was processed first and the earlierst last.
An extra benefit is the removal of the ugly hack from the Berkly days on
LFS.
In the proces, i've also replaced the various variations hand written loops
by the TAILQ_FOREACH() macro's.
to do with the security model used. Move back the call to spec_open(),
which can now return the real return value from vfs_mountedon() (EBUSY)
and not EPERM, changing semantics.
- Add a few scopes to the kernel: system, network, and machdep.
- Add a few more actions/sub-actions (requests), and start using them as
opposed to the KAUTH_GENERIC_ISSUSER place-holders.
- Introduce a basic set of listeners that implement our "traditional"
security model, called "bsd44". This is the default (and only) model we
have at the moment.
- Update all relevant documentation.
- Add some code and docs to help folks who want to actually use this stuff:
* There's a sample overlay model, sitting on-top of "bsd44", for
fast experimenting with tweaking just a subset of an existing model.
This is pretty cool because it's *really* straightforward to do stuff
you had to use ugly hacks for until now...
* And of course, documentation describing how to do the above for quick
reference, including code samples.
All of these changes were tested for regressions using a Python-based
testsuite that will be (I hope) available soon via pkgsrc. Information
about the tests, and how to write new ones, can be found on:
http://kauth.linbsd.org/kauthwiki
NOTE FOR DEVELOPERS: *PLEASE* don't add any code that does any of the
following:
- Uses a KAUTH_GENERIC_ISSUSER kauth(9) request,
- Checks 'securelevel' directly,
- Checks a uid/gid directly.
(or if you feel you have to, contact me first)
This is still work in progress; It's far from being done, but now it'll
be a lot easier.
Relevant mailing list threads:
http://mail-index.netbsd.org/tech-security/2006/01/25/0011.htmlhttp://mail-index.netbsd.org/tech-security/2006/03/24/0001.htmlhttp://mail-index.netbsd.org/tech-security/2006/04/18/0000.htmlhttp://mail-index.netbsd.org/tech-security/2006/05/15/0000.htmlhttp://mail-index.netbsd.org/tech-security/2006/08/01/0000.htmlhttp://mail-index.netbsd.org/tech-security/2006/08/25/0000.html
Many thanks to YAMAMOTO Takashi, Matt Thomas, and Christos Zoulas for help
stablizing kauth(9).
Full credit for the regression tests, making sure these changes didn't break
anything, goes to Matt Fleming and Jaime Fournier.
Happy birthday Randi! :)
fileassoc.diff adds a fileassoc_table_run() routine that allows you to
pass a callback to be called with every entry on a given mount.
veriexec.diff adds some raw device access policies: if raw disk is
opened at strict level 1, all fingerprints on this disk will be
invalidated as a safety measure. level 2 will not allow opening disk
for raw writing if we monitor it, and prevent raw writes to memory.
level 3 will not allow opening any disk for raw writing.
both update all relevant documentation.
veriexec concept is okay blymn@.
intervened by truncation.
it also fixes a deadlock. (g_glock vs pages locking order)
- uvm_vnp_setsize: modify v_size while holding v_interlock.
reviewed by Chuck Silvers.
While touching all vptofh/fhtovp functions, get rid of VFS_MAXFIDSIZ,
version the getfh(2) syscall and explicitly pass the size available in
the filehandle from userland.
Discussed on tech-kern, with lots of help from yamt (thanks!).
the splay tree has been added for these types. Fix kern/33797 by
Geoff C. Wing.
While here also fix writes the same way (probably broken for 2 years),
and properly implement KERNFS_XREAD.
The IPsec code could probably be moved out now, and use kernfs_alloctype().
- struct timeval time is gone
time.tv_sec -> time_second
- struct timeval mono_time is gone
mono_time.tv_sec -> time_uptime
- access to time via
{get,}{micro,nano,bin}time()
get* versions are fast but less precise
- support NTP nanokernel implementation (NTP API 4)
- further reading:
Timecounter Paper: http://phk.freebsd.dk/pubs/timecounter.pdf
NTP Nanokernel: http://www.eecis.udel.edu/~mills/ntp/html/kern.html
- remove GOP_SIZE_READ/GOP_SIZE_WRITE flags.
they have not been used since the change.
- ufs_balloc_range: remove code which has been no-op since the change.
thanks Konrad Schroder for explaining the original intention of the code.
- ffs_gop_size: don't extend past eof, in the case of GOP_SIZE_MEM.
otherwise genfs_getpages end up to allocate pages past eof unnecessarily.
way writes are handled: Add KERNFS_XREAD and KERNFS_FILEOP_WRITE files
operations definitions to kfsfileop, a xread function pointer to
kernfs_fileop, rename kernfs_read to kernfs_default_xread and add a
kernfs_read calling kernfs_try_fileop(KERNFS_FILEOP_READ).
Proposed on tech-kern on Feb 18 2006.
- use vmspace rather than proc or lwp where appropriate.
the latter is more natural to specify an address space.
(and less likely to be abused for random purposes.)
- fix a swdmover race.
directly to the directory containing the pid instead of pointing to
/proc/curproc, because some programs rely on calling readlink on /proc/self
to get the pid.
- add an assertion.
- call VOP_STRATEGY of underlying vnode directly, rather than
through the filesystem vnode.
- no need to set b_dev here because VOP_STRATEGY will take care of it.
This closes a hole pointed out by Thor Lancelot Simon on tech-kern ~3
years ago.
The problem was with running binaries from remote storage, where our
kernel (and Veriexec) has no control over any changes to files.
An attacker could, after the fingerprint has been verified and
program loaded to memory, inject malicious code into the backing
store on the remote storage, followed by a forced flush, causing
a page-in of the malicious data from backing store, bypassing
integrity checks.
Initial implementation by Brett Lymn.
link points to the process's current working directory, and the root
link points to the process's root directory. What else would you
expect?
For directories that are out of reach (caller is in a chroot, target
process is in a different chroot, etc), the links point to "/"
instead.
- Remove all NFS related stuff from file system specific code.
- Drop the vfs_checkexp hook and generalize it in the new nfs_check_export
function, thus removing redundancy from all file systems.
- Move all NFS export-related stuff from kern/vfs_subr.c to the new
file sys/nfs/nfs_export.c. The former was becoming large and its code
is always compiled, regardless of the build options. Using the latter,
the code is only compiled in when NFSSERVER is enabled. While doing this,
also make some functions in nfs_subs.c conditional to NFSSERVER.
- Add a new command in nfssvc(2), called NFSSVC_SETEXPORTSLIST, that takes a
path and a set of export entries. At the moment it can only clear the
exports list or append entries, one by one, but it is done in a way that
allows setting the whole set of entries atomically in the future (see the
comment in mountd_set_exports_list or in doc/TODO).
- Change mountd(8) to use the nfssvc(2) system call instead of mount(2) so
that it becomes file system agnostic. In fact, all this whole thing was
done to remove a 'XXX' block from this utility!
- Change the mount*, newfs and fsck* userland utilities to not deal with NFS
exports initialization; done internally by the kernel when initializing
the NFS support for each file system.
- Implement an interface for VFS (called VFS hooks) so that several kernel
subsystems can run arbitrary code upon receipt of specific VFS events.
At the moment, this only provides support for unmount and is used to
destroy NFS exports lists from the file systems being unmounted, though it
has room for extension.
Thanks go to yamt@, chs@, thorpej@, wrstuden@ and others for their comments
and advice in the development of this patch.
vfs.sync.delay: max time to delay syncing data
vfs.sync.filedelay: time to delay syncing files
vfs.sync.dirdelay: time to delay syncing directories
vfs.sync.metadelay: time to delay syncing metadata
Note that using a value of 0 is allowed, but it's not
recommended.
- genfs_putpages: wait for i/o completion of PG_RELEASED/PG_PAGEOUT pages by
setting "wasclean" false when encountering them.
suggested by Stephan Uphoff in PR/24596 (1).
- genfs_putpages: write protect pages when cleaning out, if
we're going to take the vnode off the syncer's queue.
uvm_fault: don't write-map pages unless its vnode is already on
the syncer's queue.
fix PR/24596 (3) but in the different way from the suggested fix.
(to keep our current behaviour, ie. not to require explicit msync.
discussed on tech-kern@.)
- genfs_putpages: don't mistakenly take a vnode off the queue
by introducing a generation number in genfs_node.
genfs_getpages: increment the generation number.
suggested by Stephan Uphoff in PR/24596 (2).
- add some assertions.
to prevent unnecessary block allocations in the case that
page size > block size.
- ufs_balloc_range: use VM_PROT_WRITE+PGO_NOBLOCKALLOC rather than
VM_PROT_READ.
current directory in curproc. Fix from Pedro Martelletto:
We cannot call vgone() from procfs_inactive() if we are coming from
vclean(). that's what's probably causing the deadlock.
into the "vfsops" link set.
- Use VFS_ATTACH() where vfsops are declared for individual file systems.
- In vfsinit(), traverse the "vfsops" link set, rather than vfs_list_initial[].
-because noone should request reads past EOF, or writes past EOF which
are not explicitely marked as file-extending (PGO_PASTEOF), turn
a boundary check into a KASSERT
approved by Chuck Silvers
1. make fileops const
2. add 2 new negative errno's to `officially' support the cloning hack:
- EDUPFD (used to overload ENODEV)
- EMOVEFD (used to overload ENXIO)
3. Created an fdclone() function to encapsulate the operations needed for
EMOVEFD, and made all cloners use it.
4. Centralize the local noop/badop fileops functions to:
fnullop_fcntl, fnullop_poll, fnullop_kqfilter, fbadop_stat
ops, not the nolock variants. Should have no real impact as according
to mkid, we only use fifo_vnodeop_entries, via fifo_vnodeop_p,
for selective operations on fifos. All the fifo users use the native
file system's locking routines.
Removes one use of genfs_nolock and friends.
PDIRUNLOCK. The whole reason we have the flag is to note (rare)
cases where we are supposed to have the parent directory locked
but don't. Permits error handling code to know what to do with
the parrent vnode (vrele() vs vput()).
through a layered file system.
Note: we don't actually support snapshots through a layered file system,
and this routine returns an error. However we: 1) have clearly documented
what needs fixing (which isn't trivial to fix) and 2) if we do fix
this, all layered file systems can take advantage of it at once.
fix for layered-file-removal. It will work for the case of accessing
and deleting a file through the layered file system. Accessing via
the layer and deleting on the underlying still won't work, nor will
accessing via complicated structures (like two umap layers over a
given file systems).
We still need VOP_UPCALL(), but this is better than things were before.
This patch has been discussed off & on for a while. This incarnation
was tested by hannken at netbsd dot org.
- Not enabled by default. Needs kernel option FFS_SNAPSHOT.
- Change parameters of ffs_blkfree.
- Let the copy-on-write functions return an error so spec_strategy
may fail if the copy-on-write fails.
- Change genfs_*lock*() to use vp->v_vnlock instead of &vp->v_lock.
- Add flag B_METAONLY to VOP_BALLOC to return indirect block buffer.
- Add a function ffs_checkfreefile needed for snapshot creation.
- Add special handling of snapshot files:
Snapshots may not be opened for writing and the attributes are read-only.
Use the mtime as the time this snapshot was taken.
Deny mtime updates for snapshot files.
- Add function transferlockers to transfer any waiting processes from
one lock to another.
- Add vfsop VFS_SNAPSHOT to take a snapshot and make it accessible through
a vnode.
- Add snapshot support to ls, fsck_ffs and dump.
Welcome to 2.0F.
Approved by: Jason R. Thorpe <thorpej@netbsd.org>
Add a new explicit `struct proc *p' argument to socreate(), sosend().
Use that argument instead of curproc. Follow-on changes to pass that
argument to socreate(), sosend(), and (*so->so_send)() calls.
These changes reviewed and independently recoded by Matt Thomas.
Changes to soreceive() and (*dom->dom_exernalize() from Matt Thomas:
pass soreceive()'s struct uio* uio->uio_procp to unp_externalize().
Eliminate curproc from unp_externalize. Also, now soreceive() uses
its uio->uio_procp value, pass that same value downward to
((pr->pru_usrreq)() calls for consistency, instead of (struct proc * )0.
Similar changes in sys/nfs to eliminate (most) uses of curproc,
either via the req-> r_procp field of a struct nfsreq *req argument,
or by passing down new explicit struct proc * arguments.
Reviewed by: Matt Thomas, posted to tech-kern.
NB: The (*pr->pru_usrreq)() change should be tested on more (all!) protocols.
and tweak lkminit_*.c (where applicable) to call them, and to call
sysctl_teardown() when being unloaded.
This consists of (1) making setup functions not be static when being
compiled as lkms (change to sys/sysctl.h), (2) making prototypes
visible for the various setup functions in header files (changes to
various header files), and (3) making simple "load" and "unload"
functions in the actual lkminit stuff.
linux_sysctl.c also needs its root exposed (ie, made not static) for
this (when built as an lkm).
1. Checking for a negative uio_offset at the beginning. This really does
not affect us in most cases because we check that later too.
2. Checking for attempts to write to init sooner and in all cases.
now uses kernfs_allocvp to map from kernfs entry to inode number,
kernfs_allocvp is now the only place where entries are mapped to inode
numbers. Also make KERNFS_FILENO not return random results for entries
not in kern_targets.
kernel and ran for a day or so. There are still some caddr_t types in
the arguments of some calls, I will do those separately (later) as
they touch a lot more of the system.
Approved by christos@NetBSD.org.
no longer use and/or need it
- removed casts from unionfs, deadfs and fdesc
(there are more to hunt down still)
- changed vfs_quotactl args argumet from caddr_t to void *
- changed vfs_quotactl structures/callers to reflect the api change
Compiled fine and ran for about a day. Approved/reviewed by
christos@netbsd.org and gimpy@netbsd.org.
ad