Commit Graph

43 Commits

Author SHA1 Message Date
perseant 9c7f8050f4 Various bug-fixes to LFS, to wit:
Kernel:

* Add runtime quantity lfs_ravail, the number of disk-blocks reserved
  for writing.  Writes to the filesystem first reserve a maximum amount
  of blocks before their write is allowed to proceed; after the blocks
  are allocated the reserved total is reduced by a corresponding amount.

  If the lfs_reserve function cannot immediately reserve the requested
  number of blocks, the inode is unlocked, and the thread sleeps until
  the cleaner has made enough space available for the blocks to be
  reserved.  In this way large files can be written to the filesystem
  (or, smaller files can be written to a nearly-full but thoroughly
  clean filesystem) and the cleaner can still function properly.

* Remove explicit switching on dlfs_minfreeseg from the kernel code; it
  is now merely a fs-creation parameter used to compute dlfs_avail and
  dlfs_bfree (and used by fsck_lfs(8) to check their accuracy).  Its
  former role is better assumed by a properly computed dlfs_avail.

* Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv.
  This prevents a panic, but, if the cleaner is feeding the filesystem
  the wrong data, you are still in a world of hurt.

* Cleanup: remove explicit references of DEV_BSIZE in favor of
  btodb()/dbtob().

lfs_cleanerd:

* Make -n mean "send N segments' blocks through a single call to
  lfs_markv".  Previously it had meant "clean N segments though N calls
  to lfs_markv, before looking again to see if more need to be cleaned".
  The new behavior gives better packing of direct data on disk with as
  little metadata as possible, largely alleviating the problem that the
  cleaner can consume more disk through inefficient use of metadata than
  it frees by moving dirty data away from clean "holes" to produce
  entirely clean segments.

* Make -b mean "read as many segments as necessary to write N segments
  of dirty data back to disk", rather than its former meaning of "read
  as many segments as necessary to free N segments worth of space".  The
  new meaning, combined with the new -n behavior described above,
  further aids in cleaning storage efficiency as entire segments can be
  written at once, using as few blocks as possible for segment summaries
  and inode blocks.

* Make the cleaner take note of segments which could not be cleaned due
  to error, and not attempt to clean them until they are entirely free
  of dirty blocks.  This prevents the case in which a cleanerd running
  with -n 1 and without -b (formerly the default) would spin trying
  repeatedly to clean a corrupt segment, while the remaining space
  filled and deadlocked the filesystem.

* Update the lfs_cleanerd manual page to describe all the options,
  including the changes mentioned here (in particular, the -b and -n
  flags were previously undocumented).

fsck_lfs:

* Check, and optionally fix, lfs_avail (to an exact figure) and
  lfs_bfree (within a margin of error) in pass 5.

newfs_lfs:

* Reduce the default dlfs_minfreeseg to 1/20 of the total segments.

* Add a warning if the sgs disklabel field is 16 (the default for FFS'
  cpg, but not usually desirable for LFS' sgs: 5--8 is a better range).

* Change the calculation of lfs_avail and lfs_bfree, corresponding to
  the kernel changes mentioned above.

mount_lfs:

* Add -N and -b options to pass corresponding -n and -b options to
  lfs_cleanerd.

* Default to calling lfs_cleanerd with "-b -n 4".


[All of these changes were largely tested in the 1.5 branch, with the
idea that they (along with previous un-pulled-up work) could be applied
to the branch while it was still in ALPHA2; however my test system has
experienced corruption on another filesystem (/dev/console has gone
missing :^), and, while I believe this unrelated to the LFS changes, I
cannot with good conscience request that the changes be pulled up.]
2000-09-09 04:49:54 +00:00
perseant 90b9d9b502 Clean up accounting of lfs_uinodes (dirty but unwritten inodes).
Make lfs_uinodes a signed quantity for debugging purposes, and set it to
zero as fs mount time.

Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED,
IN_CLEANING) in macros, and use those macros everywhere.  Make
LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know
about this.  Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-05 22:25:43 +00:00
perseant 235a4dd595 i_lfs_effnblks fixes. Put debugging printfs under #ifdef DEBUG_LFS. 2000-07-03 08:20:58 +00:00
fvdl fa78158b15 Rearrange code around getnewvnode as was already done for ffs, to avoid
locking against oneself because getnewvnode recycles a softdep-using vnode.
2000-06-30 20:45:38 +00:00
mrg 419501093a remove include of <vm/vm.h> and <uvm/uvm_extern.h> 2000-06-28 14:16:37 +00:00
perseant 39b86955ed Fixes associated with filling an LFS:
Change the space computation to appear to change the size of the *disk*
rather than the *bytes used* when more segment summaries and inode
blocks are written.  Try to estimate the amount of space that these will
take up when more files are written, so the disk size doesn't change too
much.

Regularize error returns from lfs_valloc, lfs_balloc, lfs_truncate: they
now fail entirely, rather than succeeding half-way and leaving the fs in
an inconsistent state.

Rewrite lfs_truncate, mostly stealing from ffs_truncate.  The old
lfs_truncate had difficulty truncating a large file to a non-zero size
(indirect blocks were not handled appropriately).

Unmark VDIROP on fvp after ufs_remove, ufs_rmdir, so these can be
reclaimed immediately: this vnode would not be written to disk again
anyway if the removal succeeded, and if it failed, no directory
operation occurred.

ufs_makeinode and ufs_mkdir now remove IN_ADIROP on error.
2000-06-27 20:57:11 +00:00
perseant 9235101232 fix my own typo, grr.... 2000-06-22 18:46:57 +00:00
perseant fd451352d9 Read i_ffs_gen from the version number in the Ifile during lfs_valloc,
instead of keeping it always == 1.  (The ifile version number is
increased on vfree.)  May address PR #7213, but I haven't been able to
test thoroughly enough to say for sure.
2000-06-22 18:31:49 +00:00
perseant 973ddee4e5 Update lfs_vunref for the fact that now a vnode can be locked with no
references (locked for VOP_INACTIVE at the end of vrele) and it's okay.
Check the return value of lfs_vref where appropriate.
Fixes PR #s 10285 and 10352.
2000-06-22 18:11:45 +00:00
perseant 9139045320 Protect inode free list with seglock, instead of separate lock, so that
the head of the inode free list (on the superblock) always matches the
rest of the free list (in the ifile).

Protect lfs_fragextend with seglock, to prevent the segment byte count
fudging from making its way to disk.

Don't try to inactivate dirop vnodes that are still in the middle of
their dirop (may address PR#10285).
2000-06-06 20:19:14 +00:00
perseant 61861bace5 update for IN_ACCESSED changes 2000-05-31 01:40:01 +00:00
perseant 29f616cebe Prevent dirops from getting around lfs_check and wedging the buffer cache.
All the dirop vnops now mark the inodes with a new flag, IN_ADIROP, which
is removed as soon as the dirop is done (as opposed to VDIROP which stays
until the file is written).  To address one issue raised in PR#9357.
2000-05-27 00:19:52 +00:00
perseant dac789be22 Changes to stabilize LFS. The first two of these should also apply to the
1.4 branch.

* Use a separate per-fs lock, instead of ufs_hashlock, to protect the Inode
  free list.  This seems to prevent the "lockmgr: %d, not exclusive lock holder
  %d, unlocking" message I was mis-attributing last night to an unlocked vnode
  being passed to vrele.

* Change calling semantics of lfs_ifind, to give better error reporting:
  If fed a struct buf, it can report the block number of the offending inode
  block as well as the inode number.

* Back out rev 1.10 of lfs_subr.c, since the replacement code was slightly
  uglier while being functionally identical.

* Make lfs_vunref use the same free list convention as vrele/vput, so that
  vget does not remove vnodes from a hash list they are not on.
2000-01-19 00:03:04 +00:00
perseant 32bd59cdf6 Fix error returns on lfs vnops so that locks and reference counts are
preserved.  Handle dirop accounting in lfs_vfree for this case as well.
May address PR#8823.
1999-12-15 07:19:07 +00:00
fvdl 0b1963121a Add Kirk McKusick's soft updates code to the trunk. Not enabled by
default, as the copyright on the main file (ffs_softdep.c) is such
that is has been put into gnusrc. options SOFTDEP will pull this
in. This code also contains the trickle syncer.

Bump version number to 1.4O
1999-11-15 18:49:07 +00:00
perseant fc5ecbb5a8 Back out my patch of the 8th (to address unreferenced inode problem).
Apparently this needs more thought.
1999-11-12 16:56:48 +00:00
perseant 26c34e715f If ifile blocks were written before dirops were complete, and then the
system crashed, inodes could be allocated that were not referenced.  (Though
not a serious problem, it evidences itself in phase 4 of fsck_lfs.)  Fix
this by marking if_daddr with UNASSIGNED before the inodes are actually
written; at mount time the ifile is checked for UNASSIGNED entries and
any that are found are linked back into the free list.  (The latter
functionality should move into the roll-forward agent when it materializes.)
1999-11-09 02:21:05 +00:00
perseant 3c63d0830f Address ufs_hashlock/ufs_ihashins protocol bug, discovered while doing a
post-mortem of a production machine.  Also, take the active dirop
count off of the fs and make it global (since it is measuring a global
resource) and tie the threshold value LFS_MAXDIROP to desiredvnodes.
1999-11-06 20:33:05 +00:00
perseant c588aaf55f Make changes that will allow an LFS filesystem to be used as the root
filesystem.  In particular,

- Fix mknod deadlock, described in PR 8172.
- Enable lfs_mountroot.
- Make lfs_writevnodes treat filesystems mounted on lfs device nodes properly,
  by flushing that device rather than trying to add blocks to the device inode.

This, in combination with lfs boot blocks, will allow operation of an all-lfs
system.
1999-09-03 22:48:51 +00:00
wrstuden 379a26972f Modify file systems to deal with struct lock in struct vnode. All leaf
fs's other than nfs use genfs_lock() for locking.

Modify lookup routines to set PDIRUNLOCK when they unlock the parrent.
1999-07-08 01:05:58 +00:00
tls 4d649c3adb squash some compiler warnings on debug printfs by casting to int 1999-06-17 22:22:41 +00:00
perseant a818a47169 Minor changes to the segment live bytes calculation. In particular, fixed
a bug in fragment extension that could run the count negative.  Also, don't
overcount for inodes, and don't count segment summaries.  Thus, for empty
segments the live bytes count should now be exactly zero.
1999-06-15 22:25:41 +00:00
perseant 4427f80b15 Other half of the ufs_hashlock locking fix (oops) 1999-04-16 16:23:46 +00:00
perseant 2d12a2d41d Fix locking panic on ufs_hashlock 1999-04-16 00:41:58 +00:00
perseant 6a87896854 Take out the `#ifdef USE_UFSHASH'; use ufs_hashlock to lock the inode free
list instead of free_lock.
1999-04-11 23:24:04 +00:00
mrg d2397ac5f7 completely remove Mach VM support. all that is left is the all the
header files as UVM still uses (most of) these.
1999-03-24 05:50:49 +00:00
perseant 1b8f5ea3c3 New sources should leave the LFS in a more-or-less working state. Changes
include:

	- DIROP segregation is enabled, and greater care is taken
	  to make sure that a checkpoint completes.  Fsck is not
	  needed to remount the filesystem.
	- Several checks to make sure that the LFS subsystem does not
	  overuse various resources (memory, in particular).
	- The cleaner routines, lfs_markv in particular, are completely
	  rewritten.  A buffer overflow is removed.  Greater care is taken
	  to ensure that inodes come from where lfs_cleanerd say they come
	  from (so we know nothing has changed since lfs_bmapv was called).
	- Fragment allocation is fixed, so that writes beyond end-of-file
	  do the right thing.
1999-03-10 00:20:00 +00:00
thorpej 5ecbc50348 Use DINODE_SIZE rather than sizeof(struct dinode). 1998-10-23 00:32:35 +00:00
thorpej 38cf17f475 Use the pool allocator and the "nointr" pool page allocator for LFS inodes. 1998-09-01 03:26:05 +00:00
sommerfe 7ba7fbbb23 Always include fifos; "not an option any more". 1998-06-24 20:58:44 +00:00
scottr 7171cca4b8 Protect various config(8)-generated files from inclusion while
building LKMs.  Fixes PR 5557.
1998-06-09 07:46:31 +00:00
scottr d48f258f90 Use the newly-defined opt_quota.h. 1998-06-08 04:27:50 +00:00
fvdl e5bc90f40c Merge with Lite2 + local changes 1998-03-01 02:20:01 +00:00
chs abf50cf8c3 add UVM stuff. 1998-02-07 17:28:18 +00:00
drochner 8c0a9bab00 Don't cast 64bit (off_t) file sizes to vm_offset_t (32bit on many
architectures), truncate them intelligently instead.
The truncation is done centralized in vnode_pager.c.
This prevents from wrap-over effects when parts of large (>2^32 byte) files
are mmapped.
Don't allow to mmap above the numerical range of vm_offset_t.
This is considered a temporary solution until the vm system handles the
object sizes/offsets more cleanly.
1997-07-04 20:22:09 +00:00
bouyer 76c414a957 Add support for ext2fs, this needed a few modifications to ufs/ufs/inode.h:
- added an "union inode_ext" to struct inode, for the per-fs extentions.
  For now only ext2fs uses it.
- i_din is now an union:
	union {
		struct  dinode ffs_din; /* 128 bytes of the on-disk dinode. */
		struct ext2fs_dinode e2fs_din; /* 128 bytes of the on-disk dinode. */
	} i_din
  Added a lot of #define i_ffs_* and i_e2fs_* to access the fields.
- Added two macros: FFS_ITIMES and EXT2FS_ITIMES. ITIMES calls the rigth
  macro, depending on the time of the inode. ITIMES is used where necessary,
  FFS_ITIMES and EXT2FS_ITIMES in other places.
1997-06-11 10:09:37 +00:00
mycroft 66c4e32b11 Just increment the generation count. Using the time is bogus and defeats
fsirand(8).
1997-03-10 06:18:28 +00:00
christos 90c7de0919 revert previous kprintf changes 1996-10-12 21:58:44 +00:00
christos de1b2b437e printf -> kprintf, sprintf -> ksprintf 1996-10-10 17:16:17 +00:00
pk 44af669e3d Appease gcc: unused variables if !QUOTA 1996-03-25 12:53:35 +00:00
christos 7bd9e243f3 lfs prototypes 1996-02-09 22:28:45 +00:00
cgd fccfa11af5 New RCS ID's, take two. they're more aesthecially pleasant, and use 'NetBSD' 1994-06-29 06:39:25 +00:00
mycroft 264b874c14 Update to 4.4-Lite fs code, with local changes. 1994-06-08 11:41:58 +00:00