Commit Graph

203 Commits

Author SHA1 Message Date
joff a6ef389457 If DIAGNOSTIC and the segment writer gets a badly sized buffer, panic()
instead of silently corrupting the filesystem.
2001-01-09 05:05:35 +00:00
cgd 1a1dca038e replace \<space(s)><newline> (wrong!) with \<newline> 2000-12-20 00:24:23 +00:00
perseant 32d11b86a5 Call uvm_vmp_setsize() in lfs_{fast,}vget to set initial vnode size. 2000-12-03 07:34:49 +00:00
perseant 72633be8c6 Fix typo in 'malloc' for non-MALLOCLOG case 2000-12-03 06:43:36 +00:00
perseant 2a53ff5ab9 Get rid of some old unnecessary code that cleared B_NEEDCOMMIT from buffers in
lfs_writeseg (possibly after they had been freed).

If MALLOCLOG is defined, make lfs_newbuf and lfs_freebuf pass along the
caller's file and line to _malloc and _free.
2000-12-03 05:56:27 +00:00
chs 65a9d68fda don't forget to set um_lognindir (now required by ufs_bmaparray()). 2000-12-03 05:27:51 +00:00
jdolecek bf558e3b3e only include opt_ddb.h for !LKM 2000-11-30 15:59:47 +00:00
jdolecek 734f246738 no need to include fs_lfs.h, define LFS directly 2000-11-30 15:57:35 +00:00
chs aeda8d3b77 Initial integration of the Unified Buffer Cache project. 2000-11-27 08:39:39 +00:00
perseant 0055236dda If LFS_DO_ROLLFORWARD is defined, roll forward from the older checkpoint
on mount, through the newer checkpoint and on through any newer
partial-segments that may have been written but not checkpointed because
of an intervening crash.

LFS_DO_ROLLFORWARD is not defined by default.
2000-11-27 03:33:57 +00:00
perseant 77b518b85d Use u_int32_t instead of u_long to compute LFS checksums, since the
checksum is stored in a u_int32_t.
2000-11-25 02:39:34 +00:00
perseant e4911189f1 Protect lfs_{bmapv,markv} with vfs_{un,}busy. Fix a reference/lock leak
in an error case in lfs_markv.  Change the vfs_getvfs() error to return
ENOENT, for consistency with failure of vfs_busy().

99% of this patch was from Jesse Off <joff@gci-net.com> (PR #11547).
2000-11-22 22:11:34 +00:00
perseant c398987151 More locked_queue_* and lfs_avail accounting fixes from Jesse Off
<joff@gci-net.com>.  Remove a specious btodb() in lfs_fragextend, and
count blocks shrunk or removed by VOP_TRUNCATE in lfs_avail.
2000-11-21 00:00:31 +00:00
toshii 92a17c6ecd Make buildable again.
The previous commit was a backout of rev. 1.45, which must be an accident.
2000-11-18 02:11:23 +00:00
perseant 31fc62d4e9 Correct accounting of lfs_avail, locked_queue_count, and locked_queue_bytes.
(PR #11468).  In the case of fragment allocation, check to see if enough
space is available before extending a fragment already scheduled for writing.

The locked_queue_* variables indicate the number of buffer headers and bytes,
respectively, that are unavailable to getnewbuf() because they are locked up
waiting for LFS to flush them; make sure that that is actually what we're
counting, i.e., never count malloced buffers, and always use b_bufsize instead
of b_bcount.

If DEBUG is defined, the periodic calls to lfs_countlocked will now complain
if either counter is incorrect.  (In the future lfs_countlocked will not need
to be called at all if DEBUG is not defined.)
2000-11-17 19:14:41 +00:00
perseant b880487624 Initialize the cleaner information in the Ifile from the same info from
the superblock at fs mount time, enabling the previous patch to fsck_lfs.
Patch from Jesse Off <joff@gci-net.com> (Closes PR #11470).
2000-11-14 00:42:55 +00:00
perseant a07c936a59 Remove debugging code that accidentally went in with yesterday's commit. 2000-11-13 00:24:30 +00:00
perseant c4c7b2adbb Do not needlessly dirty segment table blocks during lfs_segwrite,
preventing needless disk activity when the filesystem is idle.  (PR #10979.)
2000-11-12 07:58:36 +00:00
toshii af22f56146 Fix obsolete comments in lfs_writeinode since rev. 1.27.
New comments are mostly from perseant, with my additions.
2000-11-12 02:13:51 +00:00
toshii 0036e468ef In lfs_fastvget(), initialize i_lfs_effnblks correctly. 2000-10-21 13:53:25 +00:00
perseant 26f26aafcd Do not increment the clean segment counter, if a segment that the cleaner
is trying to clean is already clean (e.g., if two lfs_cleanerds are running
at once.)
2000-10-20 17:48:05 +00:00
perseant 7a4d35b365 In lfs_truncate, don't overcount the real blocks removed from the inode,
when deallocating a fragment that has not made it to disk yet.

Also, during dirops, give the directory vnode an extra reference in
SET_DIROP, to ensure its continued existence during SET_ENDOP, preventing
a possible NULL-dereference there.

These two changes should close PR #11064.
2000-10-14 23:22:14 +00:00
fvdl db4108490a Adapt for VOP_FSYNC parameter change. 2000-09-19 22:01:59 +00:00
perseant a477e1b98b Cast back to int32_t in LFS_EST_BFREE and LFS_EST_RSVD macros, for
consistency with their arguments.

Change the debugging printf in lfs_reserve to match, and enclose it in
#ifdef DEBUG.

Tested on alpha, arm32, sparc.
2000-09-13 00:07:56 +00:00
perseant 78ae325de3 Make this file compile on the alpha as well (use %ld and cast to long,
instead of %qd with no cast).
2000-09-12 03:22:53 +00:00
augustss 76451577e7 Make this file compile again. 2000-09-10 00:20:45 +00:00
perseant 17cb334f4f oops 2000-09-09 21:03:31 +00:00
perseant 9c7f8050f4 Various bug-fixes to LFS, to wit:
Kernel:

* Add runtime quantity lfs_ravail, the number of disk-blocks reserved
  for writing.  Writes to the filesystem first reserve a maximum amount
  of blocks before their write is allowed to proceed; after the blocks
  are allocated the reserved total is reduced by a corresponding amount.

  If the lfs_reserve function cannot immediately reserve the requested
  number of blocks, the inode is unlocked, and the thread sleeps until
  the cleaner has made enough space available for the blocks to be
  reserved.  In this way large files can be written to the filesystem
  (or, smaller files can be written to a nearly-full but thoroughly
  clean filesystem) and the cleaner can still function properly.

* Remove explicit switching on dlfs_minfreeseg from the kernel code; it
  is now merely a fs-creation parameter used to compute dlfs_avail and
  dlfs_bfree (and used by fsck_lfs(8) to check their accuracy).  Its
  former role is better assumed by a properly computed dlfs_avail.

* Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv.
  This prevents a panic, but, if the cleaner is feeding the filesystem
  the wrong data, you are still in a world of hurt.

* Cleanup: remove explicit references of DEV_BSIZE in favor of
  btodb()/dbtob().

lfs_cleanerd:

* Make -n mean "send N segments' blocks through a single call to
  lfs_markv".  Previously it had meant "clean N segments though N calls
  to lfs_markv, before looking again to see if more need to be cleaned".
  The new behavior gives better packing of direct data on disk with as
  little metadata as possible, largely alleviating the problem that the
  cleaner can consume more disk through inefficient use of metadata than
  it frees by moving dirty data away from clean "holes" to produce
  entirely clean segments.

* Make -b mean "read as many segments as necessary to write N segments
  of dirty data back to disk", rather than its former meaning of "read
  as many segments as necessary to free N segments worth of space".  The
  new meaning, combined with the new -n behavior described above,
  further aids in cleaning storage efficiency as entire segments can be
  written at once, using as few blocks as possible for segment summaries
  and inode blocks.

* Make the cleaner take note of segments which could not be cleaned due
  to error, and not attempt to clean them until they are entirely free
  of dirty blocks.  This prevents the case in which a cleanerd running
  with -n 1 and without -b (formerly the default) would spin trying
  repeatedly to clean a corrupt segment, while the remaining space
  filled and deadlocked the filesystem.

* Update the lfs_cleanerd manual page to describe all the options,
  including the changes mentioned here (in particular, the -b and -n
  flags were previously undocumented).

fsck_lfs:

* Check, and optionally fix, lfs_avail (to an exact figure) and
  lfs_bfree (within a margin of error) in pass 5.

newfs_lfs:

* Reduce the default dlfs_minfreeseg to 1/20 of the total segments.

* Add a warning if the sgs disklabel field is 16 (the default for FFS'
  cpg, but not usually desirable for LFS' sgs: 5--8 is a better range).

* Change the calculation of lfs_avail and lfs_bfree, corresponding to
  the kernel changes mentioned above.

mount_lfs:

* Add -N and -b options to pass corresponding -n and -b options to
  lfs_cleanerd.

* Default to calling lfs_cleanerd with "-b -n 4".


[All of these changes were largely tested in the 1.5 branch, with the
idea that they (along with previous un-pulled-up work) could be applied
to the branch while it was still in ALPHA2; however my test system has
experienced corruption on another filesystem (/dev/console has gone
missing :^), and, while I believe this unrelated to the LFS changes, I
cannot with good conscience request that the changes be pulled up.]
2000-09-09 04:49:54 +00:00
perseant 988a012d50 Change dlfs_dmeta and dlfs_avail to signed quantities, to prevent
underflow errors, visible in userland as impossibly high values
returned from df(1).
2000-09-09 04:18:28 +00:00
perseant 8cb6723a92 Fix a buffer-cache corrupting bug in lfs_writeseg, where brelse could
be improperly used on an already-queued buffer.
2000-09-09 04:13:43 +00:00
perseant 4446817aca Make sure to unmark B_DELWRI on blocks freed due to truncation to a non-zero
file length.  Should fix PR #s 10551 and 10831.
2000-09-09 03:47:05 +00:00
thorpej f9ba0345f1 XXX Use of hzto() return value needs to be double-checked here. 2000-07-13 17:35:03 +00:00
perseant 90b9d9b502 Clean up accounting of lfs_uinodes (dirty but unwritten inodes).
Make lfs_uinodes a signed quantity for debugging purposes, and set it to
zero as fs mount time.

Enclose setting/clearing of the dirty flags (IN_MODIFIED, IN_ACCESSED,
IN_CLEANING) in macros, and use those macros everywhere.  Make
LFS_ITIMES use these macros; updated the ITIMES macro in inode.h to know
about this.  Make ufs_getattr use ITIMES instead of FFS_ITIMES.
2000-07-05 22:25:43 +00:00
perseant bc303e52cb Fix errors observed while trying to fill the filesystem with yesterday's
fixes:

- Write copies of bfree and avail in the CLEANERINFO block, so the
  cleaner doesn't have to guess which superblock has the current
  information (if indeed any do).

- Tighten up accounting of lfs_avail (more needs to be done).

- When cleansing indirect blocks of UNWRITTEN, make sure not to mark
  them clean, since they'll need to be rewritten later.
2000-07-04 22:30:37 +00:00
perseant eccd5fb47a Fix i_ffs_blocks in fragment extension case where fragment has not yet
been written to disk.
2000-07-03 20:12:42 +00:00
fvdl 7c1d5ec58f Correct typo in previous. 2000-07-03 18:22:10 +00:00
perseant 235a4dd595 i_lfs_effnblks fixes. Put debugging printfs under #ifdef DEBUG_LFS. 2000-07-03 08:20:58 +00:00
perseant ef2da50400 Allow the number of free segments reserved for the cleaner to be
parametrized in the filesystem, defaulting to MIN_FREE_SEGS = 2 but set
to something more reasonable at newfs_lfs time.

Note the number of blocks that have been scheduled for writing but which
are not yet on disk in an inode extension, i_lfs_effnblks.  Move
i_ffs_effnlink out of the ffs extension and onto the main inode, since
it's used all over the shared code and the lfs extension would clobber
it.

At inode write time, indirect blocks and inode-held blocks of inodes
that have i_lfs_effnblks != i_ffs_blocks are cleansed of UNWRITTEN disk
addresses, so that these never make it to disk.
2000-07-03 01:45:46 +00:00
perseant db557257e5 Move SET_ENDOP after vrele to avoid deactivating vnode twice, if
SET_ENDOP triggers a write.
2000-07-01 19:03:57 +00:00
fvdl fa78158b15 Rearrange code around getnewvnode as was already done for ffs, to avoid
locking against oneself because getnewvnode recycles a softdep-using vnode.
2000-06-30 20:45:38 +00:00
mrg 419501093a remove include of <vm/vm.h> and <uvm/uvm_extern.h> 2000-06-28 14:16:37 +00:00
mrg 91cc436b9e <vm/vm.h> -> <uvm/uvm_extern.h> 2000-06-28 14:11:33 +00:00
perseant 39b86955ed Fixes associated with filling an LFS:
Change the space computation to appear to change the size of the *disk*
rather than the *bytes used* when more segment summaries and inode
blocks are written.  Try to estimate the amount of space that these will
take up when more files are written, so the disk size doesn't change too
much.

Regularize error returns from lfs_valloc, lfs_balloc, lfs_truncate: they
now fail entirely, rather than succeeding half-way and leaving the fs in
an inconsistent state.

Rewrite lfs_truncate, mostly stealing from ffs_truncate.  The old
lfs_truncate had difficulty truncating a large file to a non-zero size
(indirect blocks were not handled appropriately).

Unmark VDIROP on fvp after ufs_remove, ufs_rmdir, so these can be
reclaimed immediately: this vnode would not be written to disk again
anyway if the removal succeeded, and if it failed, no directory
operation occurred.

ufs_makeinode and ufs_mkdir now remove IN_ADIROP on error.
2000-06-27 20:57:11 +00:00
perseant d8584d7769 From John Evans <jevans@cray.com>: use datosn() to convert to segment
number, when remarking the current segment ACTIVE.  See PR #10463.
2000-06-27 20:00:03 +00:00
perseant 9235101232 fix my own typo, grr.... 2000-06-22 18:46:57 +00:00
perseant fd451352d9 Read i_ffs_gen from the version number in the Ifile during lfs_valloc,
instead of keeping it always == 1.  (The ifile version number is
increased on vfree.)  May address PR #7213, but I haven't been able to
test thoroughly enough to say for sure.
2000-06-22 18:31:49 +00:00
perseant 973ddee4e5 Update lfs_vunref for the fact that now a vnode can be locked with no
references (locked for VOP_INACTIVE at the end of vrele) and it's okay.
Check the return value of lfs_vref where appropriate.
Fixes PR #s 10285 and 10352.
2000-06-22 18:11:45 +00:00
perseant 4be4b8ad47 Don't try to inactivate dirop vnodes that are still in the middle of
their dirop.
2000-06-06 22:56:54 +00:00
perseant 9139045320 Protect inode free list with seglock, instead of separate lock, so that
the head of the inode free list (on the superblock) always matches the
rest of the free list (in the ifile).

Protect lfs_fragextend with seglock, to prevent the segment byte count
fudging from making its way to disk.

Don't try to inactivate dirop vnodes that are still in the middle of
their dirop (may address PR#10285).
2000-06-06 20:19:14 +00:00
fredb ef3785c36a Make this build. (Balance parenthesis. 2000-05-31 03:37:34 +00:00