through a layered file system.
Note: we don't actually support snapshots through a layered file system,
and this routine returns an error. However we: 1) have clearly documented
what needs fixing (which isn't trivial to fix) and 2) if we do fix
this, all layered file systems can take advantage of it at once.
fix for layered-file-removal. It will work for the case of accessing
and deleting a file through the layered file system. Accessing via
the layer and deleting on the underlying still won't work, nor will
accessing via complicated structures (like two umap layers over a
given file systems).
We still need VOP_UPCALL(), but this is better than things were before.
This patch has been discussed off & on for a while. This incarnation
was tested by hannken at netbsd dot org.
- Not enabled by default. Needs kernel option FFS_SNAPSHOT.
- Change parameters of ffs_blkfree.
- Let the copy-on-write functions return an error so spec_strategy
may fail if the copy-on-write fails.
- Change genfs_*lock*() to use vp->v_vnlock instead of &vp->v_lock.
- Add flag B_METAONLY to VOP_BALLOC to return indirect block buffer.
- Add a function ffs_checkfreefile needed for snapshot creation.
- Add special handling of snapshot files:
Snapshots may not be opened for writing and the attributes are read-only.
Use the mtime as the time this snapshot was taken.
Deny mtime updates for snapshot files.
- Add function transferlockers to transfer any waiting processes from
one lock to another.
- Add vfsop VFS_SNAPSHOT to take a snapshot and make it accessible through
a vnode.
- Add snapshot support to ls, fsck_ffs and dump.
Welcome to 2.0F.
Approved by: Jason R. Thorpe <thorpej@netbsd.org>
Add a new explicit `struct proc *p' argument to socreate(), sosend().
Use that argument instead of curproc. Follow-on changes to pass that
argument to socreate(), sosend(), and (*so->so_send)() calls.
These changes reviewed and independently recoded by Matt Thomas.
Changes to soreceive() and (*dom->dom_exernalize() from Matt Thomas:
pass soreceive()'s struct uio* uio->uio_procp to unp_externalize().
Eliminate curproc from unp_externalize. Also, now soreceive() uses
its uio->uio_procp value, pass that same value downward to
((pr->pru_usrreq)() calls for consistency, instead of (struct proc * )0.
Similar changes in sys/nfs to eliminate (most) uses of curproc,
either via the req-> r_procp field of a struct nfsreq *req argument,
or by passing down new explicit struct proc * arguments.
Reviewed by: Matt Thomas, posted to tech-kern.
NB: The (*pr->pru_usrreq)() change should be tested on more (all!) protocols.
and tweak lkminit_*.c (where applicable) to call them, and to call
sysctl_teardown() when being unloaded.
This consists of (1) making setup functions not be static when being
compiled as lkms (change to sys/sysctl.h), (2) making prototypes
visible for the various setup functions in header files (changes to
various header files), and (3) making simple "load" and "unload"
functions in the actual lkminit stuff.
linux_sysctl.c also needs its root exposed (ie, made not static) for
this (when built as an lkm).
1. Checking for a negative uio_offset at the beginning. This really does
not affect us in most cases because we check that later too.
2. Checking for attempts to write to init sooner and in all cases.
now uses kernfs_allocvp to map from kernfs entry to inode number,
kernfs_allocvp is now the only place where entries are mapped to inode
numbers. Also make KERNFS_FILENO not return random results for entries
not in kern_targets.
kernel and ran for a day or so. There are still some caddr_t types in
the arguments of some calls, I will do those separately (later) as
they touch a lot more of the system.
Approved by christos@NetBSD.org.
no longer use and/or need it
- removed casts from unionfs, deadfs and fdesc
(there are more to hunt down still)
- changed vfs_quotactl args argumet from caddr_t to void *
- changed vfs_quotactl structures/callers to reflect the api change
Compiled fine and ran for about a day. Approved/reviewed by
christos@netbsd.org and gimpy@netbsd.org.
case, we tear down the node-specific storage as if there were no more open
users. As vclean() will VT_NON the vnode before anyone else will get access
to the vnode, this is our last chance.
Fixes memory leak in revoke(2) path noticed by tedu at openbsd dot org.
called with every buffer written through spec_strategy().
Used by fss(4). Future file-system-internal snapshots will need them too.
Welcome to 1.6ZK
Approved by: Jason R. Thorpe <thorpej@netbsd.org>
VOP_STRATEGY(bp) is replaced by one of two new functions:
- VOP_STRATEGY(vp, bp) Call the strategy routine of vp for bp.
- DEV_STRATEGY(bp) Call the d_strategy routine of bp->b_dev for bp.
DEV_STRATEGY(bp) is used only for block-to-block device situations.
Uses a hook in spec_strategy() to save data written from a mounted
file system to its block device and a hook in dounmount().
Not enabled by default in any kernel config.
Approved by: Frank van der Linden <fvdl@netbsd.org>
Gone are the old kern_sysctl(), cpu_sysctl(), hw_sysctl(),
vfs_sysctl(), etc, routines, along with sysctl_int() et al. Now all
nodes are registered with the tree, and nodes can be added (or
removed) easily, and I/O to and from the tree is handled generically.
Since the nodes are registered with the tree, the mapping from name to
number (and back again) can now be discovered, instead of having to be
hard coded. Adding new nodes to the tree is likewise much simpler --
the new infrastructure handles almost all the work for simple types,
and just about anything else can be done with a small helper function.
All existing nodes are where they were before (numerically speaking),
so all existing consumers of sysctl information should notice no
difference.
PS - I'm sorry, but there's a distinct lack of documentation at the
moment. I'm working on sysctl(3/8/9) right now, and I promise to
watch out for buses.
accepted. However, this time this behavor is not the default. Instead
it must enabled by using the LOCAL_CONNWAIT socket option on either the
connecting or accepting socket.
to the session and pgrp from the tty. The way that the console is
handled means that the vrele() may not actually do the final close
on the tty itself.
file system.
The function vfs_write_suspend stops all new write operations to a file
system, allows any file system modifying system calls already in progress
to complete, then sync's the file system to disk and returns. The
function vfs_write_resume allows the suspended write operations to
complete.
From FreeBSD with slight modifications.
Approved by: Frank van der Linden <fvdl@netbsd.org>
are opened (was always done for block devices).
This means that fstat will report the partition size and hence newfs
needn't grovel into the disklabel to find the filesystem size.
to have KFS or PFS differentiators. Further I have wrapped the enum in
procfs in "#ifdef _KERNEL" as it is done in kernfs.
To see the discussion go to http://mail-index.NetBSD.org/tech-kern/2003/09/
and look for "Mismatched enums in include files" in the list.
genfs_getpages() can read in more blocks than it should due to faked filesize
of lfs_gop_size(). it's a security problem and it makes gcc3 "internal error"
to fix this,
- in genfs_getpages(), always calculate diskeof and memeof separately
so that filesystems (in this case, lfs) can use different strategies
for them.
- introduce GOP_SIZE_MEM flag and use it to request in-core filesize.
(it was an intention of GOP_SIZE_READ,
but after the above change _READ is not a straightforward name)
after this, no one uses GOP_SIZE_{READ,WRITE} anymore but leave them for now.
Handy because ls(1) helpfully converts the time to human-readable
format when printing, and because shell tools like "test -nt" and
"find -newer" can be used against it.
"Inspired" by a discussion about removing lockfiles older than the
last reboot, and Al Crooks' handy observation that a close
approximation can be found with /var/run/dmesg.boot
While here, notice that a lot of the kernfs structures and naming
changed suddenly, and though it seems a clear improvement, there was no
mention in commit logs.
* Remove the "lwp *" argument that was added to vget(). Turns out
that nothing actually used it!
* Remove the "lwp *" arguments that were added to VFS_ROOT(), VFS_VGET(),
and VFS_FHTOVP(); all they did was pass it to vget() (which, as noted
above, didn't use it).
* Remove all of the "lwp *" arguments to internal functions that were added
just to appease the above.
be inserted into ktrace records. The general change has been to replace
"struct proc *" with "struct lwp *" in various function prototypes, pass
the lwp through and use l_proc to get the process pointer when needed.
Bump the kernel rev up to 1.6V
use malloc/free for array of pointers to vm_page.
otherwise, use on-stack array as used to.
this change fixes assertion failure when nfsd gets a big read request
that isn't aligned with filesystem block.
discussed on tech-kern.
first step towards per-device MAXPHYS, and has the beneficial side effect
of allowing clustering to MAXPHYS even on systems that need to run with
a reduced MAXBSIZE to get more metadata buffers.
- Under chroot it displays only the visible filesystems with appropriate paths.
- The statfs f_mntonname gets adjusted to contain the real path from root.
- While was there, fixed a bug in ext2fs, locking problems with vfs_getfsstat(),
and factored out some of the vfsop statfs() code to copy_statfs_info(). This
fixes the problem where some filesystems forgot to set fsid.
- Made coda look more like a normal fs.
are vput()/vrele()d as necessary - some filesystems did use the wrong
one for some ops, and it's just safer to not take the chance
based on suggestion by Bill Studenmund
which uses WILLPUT for member which may be NULL
handle correctly dvp == vp case for WILLPUT members, so this works
for vop_remove, vop_rename
thanks Bill Studenmund for code&comments on this
before the reader woke up - this made the reader loop again, waiting
for another writer, even though there was input available.
Thanks to Jaromir for spotting the real cause and sugesting a solution.
This should fix PR port-sparc64/20283.
(there are still some details to work out) but expect that to go
away soon. To support these basic changes (creation of lfs_putpages,
lfs_gop_write, mods to lfs_balloc) several other changes were made, to
wit:
* Create a writer daemon kernel thread whose purpose is to handle page
writes for the pagedaemon, but which also takes over some of the
functions of lfs_check(). This thread is started the first time an
LFS is mounted.
* Add a "flags" parameter to GOP_SIZE. Current values are
GOP_SIZE_READ, meaning that the call should return the size of the
in-core version of the file, and GOP_SIZE_WRITE, meaning that it
should return the on-disk size. One of GOP_SIZE_READ or
GOP_SIZE_WRITE must be specified.
* Instead of using malloc(...M_WAITOK) for everything, reserve enough
resources to get by and use malloc(...M_NOWAIT), using the reserves if
necessary. Use the pool subsystem for structures small enough that
this is feasible. This also obsoletes LFS_THROTTLE.
And a few that are not strictly necessary:
* Moves the LFS inode extensions off onto a separately allocated
structure; getting closer to LFS as an LKM. "Welcome to 1.6O."
* Unified GOP_ALLOC between FFS and LFS.
* Update LFS copyright headers to correct values.
* Actually cast to unsigned in lfs_shellsort, like the comment says.
* Keep track of which segments were empty before the previous
checkpoint; any segments that pass two checkpoints both dirty and
empty can be summarily cleaned. Do this. Right now lfs_segclean
still works, but this should be turned into an effectless
compatibility syscall.
* do not set *vpp unless successful, otherwise we'd trigger
DIAGNOSTIC panic in lookup(9) on error return
* on error, make sure to free malloc'ed memory and ungetnewvnode() the
previously acquired vnode
this fixes panic on 'tail -f <file> &; ls -l /proc/$!/fd' reported by
Andrew Brown
fix reviewed by Christos Zoulas
malloc types into a structure, a pointer to which is passed around,
instead of an int constant. Allow the limit to be adjusted when the
malloc type is defined, or with a function call, as suggested by
Jonathan Stone.
- Is it ok to convert DTYPE_PIPE to VFIFO and DTYPE_SOCKET to VSOCK?
- XXX: Avoid locking issue in ls -Rl /proc by avoiding curproc
- Does I/O to pipes work?
- XXX: Are there security implications?
kqueue provides a stateful and efficient event notification framework
currently supported events include socket, file, directory, fifo,
pipe, tty and device changes, and monitoring of processes and signals
kqueue is supported by all writable filesystems in NetBSD tree
(with exception of Coda) and all device drivers supporting poll(2)
based on work done by Jonathan Lemon for FreeBSD
initial NetBSD port done by Luke Mewburn and Jason Thorpe
This merge changes the device switch tables from static array to
dynamically generated by config(8).
- All device switches is defined as a constant structure in device drivers.
- The new grammer ``device-major'' is introduced to ``files''.
device-major <prefix> char <num> [block <num>] [<rules>]
- All device major numbers must be listed up in port dependent majors.<arch>
by using this grammer.
- Added the new naming convention.
The name of the device switch must be <prefix>_[bc]devsw for auto-generation
of device switch tables.
- The backward compatibility of loading block/character device
switch by LKM framework is broken. This is necessary to convert
from block/character device major to device name in runtime and vice versa.
- The restriction to assign device major by LKM is completely removed.
We don't need to reserve LKM entries for dynamic loading of device switch.
- In compile time, device major numbers list is packed into the kernel and
the LKM framework will refer it to assign device major number dynamically.
has been VOP_OPEN()'d. if the fifo is being accessed via a layered fs,
v_usecount is always one (representing the hold by the layered vnode)
regardless of how many times the vnode has been opened. instead, keep a
separate counter for opens. fixes PR 17195 and probably 17724.
gets reset properly when the old parent exits before the child. A flag
is set in old parent process when the child is reparented in ptrace(2).
If it's set when process is exiting, all running processes have their
'old parent process' pointer checked and reset if appropriate. Also
change to use 'struct proc *' pointer directly, rather than pid_t.
This fixes security/14444 by David Sainty.
Reviewed by Christos Zoulas.
Fixes kern/10278 by IWAMOTO Toshihiro, though implemented different way.
While here, clean up some int vs. size_t confusion, make
kernfs_x{read|write}() static and g/c some #if 0 stuff.
operation if more than one LIST_REMOVE happens on interrupt, so both the test
for VONWORKLIST and the LIST_REMOVE(vp, v_synclist) need to be in splbio().
* fdesc_attr(): don't panic for 'unknown' descriptor types, rather use
(*fp->f_ops->fo_stat)() hook, as for DTYPE_SOCKET and DTYPE_PIPE
XXX perhaps use different vnode type than VBAD for these?
* fdesc_setattr(): just return 0 regardless of type, rather than paniccing
for 'unknown' descriptor types
indicating an unhandled "command". ERESTART is -1, which can lead to
confusion. ERESTART has been moved to -3 and EPASSTHROUGH has been
placed at -4. No ioctl code should now return -1 anywhere. The
ioctl() system call is now properly restartable.
- when yielding the cpu while using the vnode's page list, use a marker page
to keep our place in the list (like the other cases where we drop the lock).
- wait until no one else has the page busy before deciding if the page needs
to be cleaned. a page will be dirty while it's being initialized but will
be marked clean before PG_BUSY is cleared.
both found by enami.
not going to wait for. this doesn't matter for real devices since we call
VOP_STRATEGY() directly, but NFS uses this flag to decide whether or not
to hand the buffer off to an nfsiod thread.
(__HAVE_PTRACE_MACHDEP) and procfs (__HAVE_PROCFS_MACHDEP).
These changes will allow platforms like x86 (XMM) and PowerPC
(AltiVec) to export extended register sets in a sane manner.
* Use __HAVE_PTRACE_MACHDEP to export x86 XMM registers (standard
FP + SSE/SSE2) using PT_{GET,SET}XMMREGS (in the machdep
ptrace request space).
* Use __HAVE_PROCFS_MACHDEP to export x86 XMM registers via
/proc/N/xmmregs in procfs.
other code has this limit. Also while I am here, convert the magic 16 into
a #define constant and use it in the appropriate places. This is a temporary
fix, since all this read-ahead business is XXXUBC anyway.
- remove special treatment of pager_map mappings in pmaps. this is
required now, since I've removed the globals that expose the address range.
pager_map now uses pmap_kenter_pa() instead of pmap_enter(), so there's
no longer any need to special-case it.
- eliminate struct uvm_vnode by moving its fields into struct vnode.
- rewrite the pageout path. the pager is now responsible for handling the
high-level requests instead of only getting control after a bunch of work
has already been done on its behalf. this will allow us to UBCify LFS,
which needs tighter control over its pages than other filesystems do.
writing a page to disk no longer requires making it read-only, which
allows us to write wired pages without causing all kinds of havoc.
- use a new PG_PAGEOUT flag to indicate that a page should be freed
on behalf of the pagedaemon when it's unlocked. this flag is very similar
to PG_RELEASED, but unlike PG_RELEASED, PG_PAGEOUT can be cleared if the
pageout fails due to eg. an indirect-block buffer being locked.
this allows us to remove the "version" field from struct vm_page,
and together with shrinking "loan_count" from 32 bits to 16,
struct vm_page is now 4 bytes smaller.
- no longer use PG_RELEASED for swap-backed pages. if the page is busy
because it's being paged out, we can't release the swap slot to be
reallocated until that write is complete, but unlike with vnodes we
don't keep a count of in-progress writes so there's no good way to
know when the write is done. instead, when we need to free a busy
swap-backed page, just sleep until we can get it busy ourselves.
- implement a fast-path for extending writes which allows us to avoid
zeroing new pages. this substantially reduces cpu usage.
- encapsulate the data used by the genfs code in a struct genfs_node,
which must be the first element of the filesystem-specific vnode data
for filesystems which use genfs_{get,put}pages().
- eliminate many of the UVM pagerops, since they aren't needed anymore
now that the pager "put" operation is a higher-level operation.
- enhance the genfs code to allow NFS to use the genfs_{get,put}pages
instead of a modified copy.
- clean up struct vnode by removing all the fields that used to be used by
the vfs_cluster.c code (which we don't use anymore with UBC).
- remove kmem_object and mb_object since they were useless.
instead of allocating pages to these objects, we now just allocate
pages with no object. such pages are mapped in the kernel until they
are freed, so we can use the mapping to find the page to free it.
this allows us to remove splvm() protection in several places.
The sum of all these changes improves write throughput on my
decstation 5000/200 to within 1% of the rate of NetBSD 1.5
and reduces the elapsed time for "make release" of a NetBSD 1.5
source tree on my 128MB pc to 10% less than a 1.5 kernel took.
adjusted via sysctl. file systems that have hash tables which are
sized based on the value of this variable now resize those hash tables
using the new value. the max number of FFS softdeps is also recalculated.
convert various file systems to use the <sys/queue.h> macros for
their hash tables.
the lower layer needs to have control over that flag.
that didn't solve the whole problem that it was trying to solve anyway.
(the issue is that if we have create mappings to the lower layer,
we need to get rid of those when we copy the file to the upper layer.)
we'll have to figure out some other way to handle this.
between creation of a file descriptor and close(2) when using kernel
assisted threads. What we do is stick descriptors in the table, but
mark them as "larval". This causes essentially everything to treat
it as a non-existent descriptor, except for fdalloc(), which sees a
filled slot so that it won't (incorrectly) allocate it again. When
a descriptor is fully constructed, the code that has constructed it
marks it as "mature" (which actually clears the "larval" flag), and
things continue to work as normal.
While here, gather all the code that gets a descriptor from the table
into a fd_getfile() function, and call it, rather than having the
same (sometimes incorrect) code copied all over the place.
to resolve a write fault. fixes PR 13201.
also, be sure to allocate blocks for write faults to holes even if
the page is already in memory. fixes PR 13189.
- We need to skip PGO_DONTCARE page also.
- ``npages'' returned by VOP_GETPAGES for lower vp doesn't count
those pages in this case. So, just loop ``npages'' times is
insufficient. Loop while there is real pages instead.
callers and appropriate routines to cope. This makes fo_stat more
consistent with rest of fileops routines and also makes the fo_stat
match FreeBSD as an added bonus.
Discussed with Luke Mewburn on tech-kern@.
the mapping is:
VM_PAGER_OK 0
VM_PAGER_BAD <unused>
VM_PAGER_FAIL <unused>
VM_PAGER_PEND 0 (see below)
VM_PAGER_ERROR EIO
VM_PAGER_AGAIN EAGAIN
VM_PAGER_UNLOCK EBUSY
VM_PAGER_REFAULT ERESTART
for async i/o requests, it used to be possible for the request to
be convert to sync, and the pager would return VM_PAGER_OK or VM_PAGER_PEND
to indicate whether the caller should perform post-i/o cleanup.
this is no longer allowed; pagers must now return 0 to indicate that
the async i/o was successfully started, and the caller never needs to
worry about doing the post-i/o cleanup.
setattr calls on underlying vnodes the same as sockets and just return 0.
This whole thing needs to be gutted and replaced with either fall throughs
to specfs (the attr forwarding is just bizarre and leads to weird crap like
the above truncation problems), or better yet a real cloning device node.
which we disallow creation of page cache pages) and its on-disk EOF
(which marks the offset at which there is not (yet) data on disk that
we need to read when creating pages). for requests with PGO_PASTEOF,
the in-memory EOF maybe be much larger than the on-disk EOF.
- in genfs_getpages(), unbusy any pages that we don't free in the error path.
- in genfs_putpages(), if we get a bmap error, record that in the master buf.
- in the cases where we skip over the i/o loop, increment npages by ridx
so that when the cleanup code starts processing the pgs array at index 0
it'll actually process all of the pages.
- process the PG_RELEASED flag when unbusying pages.
- add some missing MP locking.
- use MIN() and MAX() instead of min() and max() since the latter are
functions which take arguments of type "int" but we call them with
values of type "off_t", so the values could be truncated.
- in the PGO_PASTEOF case, use the larger of the current file size and the
end of the requested range of pages as the file size for this request.
this fixes some problems with sparsing writes to large offsets.
wrstuden