Commit Graph

58 Commits

Author SHA1 Message Date
wiz 1fd7eeefcd "than" instead of "then". 2001-11-21 19:14:19 +00:00
perseant 0a9ceae750 fix printf format on alpha 2001-07-18 06:24:38 +00:00
perseant cfe2897c6e Handle segment 0 properly, if its offset is different from other segments
because of the disklabel.

Fix a problem with inode block handling that sometimes caused the wrong
blocks to be read, causing either cleaning failures or panics with v2 file
systems.
2001-07-18 05:46:43 +00:00
perseant 4e3fced95b Merge the short-lived perseant-lfsv2 branch into the trunk.
Kernels and tools understand both v1 and v2 filesystems; newfs_lfs
generates v2 by default.  Changes for the v2 layout include:

- Segments of non-PO2 size and arbitrary block offset, so these can be
  matched to convenient physical characteristics of the partition (e.g.,
  stripe or track size and offset).

- Address by fragment instead of by disk sector, paving the way for
  non-512-byte-sector devices.  In theory fragments can be as large
  as you like, though in reality they must be smaller than MAXBSIZE in size.

- Use serial number and filesystem identifier to ensure that roll-forward
  doesn't get old data and think it's new.  Roll-forward is enabled for
  v2 filesystems, though not for v1 filesystems by default.

- The inode free list is now a tailq, paving the way for undelete (undelete
  is not yet implemented, but can be without further non-backwards-compatible
  changes to disk structures).

- Inode atime information is kept in the Ifile, instead of on the inode;
  that is, the inode is never written *just* because atime was changed.
  Because of this the inodes remain near the file data on the disk, rather
  than wandering all over as the disk is read repeatedly.  This speeds up
  repeated reads by a small but noticeable amount.

Other changes of note include:

- The ifile written by newfs_lfs can now be of arbitrary length, it is no
  longer restricted to a single indirect block.

- Fixed an old bug where ctime was changed every time a vnode was created.
  I need to look more closely to make sure that the times are only updated
  during write(2) and friends, not after-the-fact during a segment write,
  and certainly not by the cleaner.
2001-07-13 20:30:18 +00:00
christos 75ac9bb540 remove redundant declarations. 2001-02-04 22:12:47 +00:00
cgd 9cfe468c74 avoid C sequence point issues warned about by development version of gcc. 2001-01-16 02:41:17 +00:00
lukem 3f963260b9 be more consistent about syslog usage. now it's more like:
err		fatal errors
	warning		warnings
	info		status messages (-d), stats on SIGxxx
	debug		debug messages (-d), debug stats
2001-01-10 01:13:54 +00:00
joff d68ab23851 Don't qsort() by the segcreate field. Prevents potentially serious filesystem
corruption if the clock jumps backwards.
2001-01-09 04:31:18 +00:00
lukem de3e6adaf6 use more standard %ll_ in favour of %q_ 2001-01-04 17:24:35 +00:00
perseant 59ca5b76e4 Don't "compress" segment data if we were using mmap instead of malloc/copy
to read the segment.
2000-11-23 23:01:55 +00:00
perseant c019319ae3 Check for ENOENT return from lfs_{bmapv,markv} and do the right thing with it,
avoiding needless looping (possibly infinite looping) on certain kinds of
errors.

Get rid of erroneous free() in error return from add_segment.

Patch from Jesse Off <joff@gci-net.com> (PR #11547).
2000-11-22 22:17:39 +00:00
perseant 9683b76b99 Try to prevent running more than one active cleaner on a filesystem at a time.
Let lfs_cleanerd record its pid in /var/run like other daemons.  Make
mount_lfs not start another cleaner when updating the mount, unless it is
being upgraded from read-only to read-write; when downgrading to read-only,
kill the cleaner using the recorded pids.
2000-11-13 22:12:49 +00:00
perseant d5e76bbac2 Don't hold every segment that is being cleaned in memory in its entirety;
instead, if the segment doesn't have many live blocks, copy them to a
more appropriately sized chunk of memory and release the original.

This should prevent the cleaner from distending itself when cleaning many
segments with only one or two live blocks each, as when using the "-b" option.
2000-11-11 22:40:13 +00:00
hubertf dca9c22e0c in SEE also, xref newfs_lfs(8), sort entries and seperate by "," 2000-11-08 19:45:08 +00:00
perseant d9cc43d8a3 Fix memory leak described in PR #11094 (patch from Jesse Off
<joff@gci-net.com>).
2000-11-03 17:52:55 +00:00
perseant 9c7f8050f4 Various bug-fixes to LFS, to wit:
Kernel:

* Add runtime quantity lfs_ravail, the number of disk-blocks reserved
  for writing.  Writes to the filesystem first reserve a maximum amount
  of blocks before their write is allowed to proceed; after the blocks
  are allocated the reserved total is reduced by a corresponding amount.

  If the lfs_reserve function cannot immediately reserve the requested
  number of blocks, the inode is unlocked, and the thread sleeps until
  the cleaner has made enough space available for the blocks to be
  reserved.  In this way large files can be written to the filesystem
  (or, smaller files can be written to a nearly-full but thoroughly
  clean filesystem) and the cleaner can still function properly.

* Remove explicit switching on dlfs_minfreeseg from the kernel code; it
  is now merely a fs-creation parameter used to compute dlfs_avail and
  dlfs_bfree (and used by fsck_lfs(8) to check their accuracy).  Its
  former role is better assumed by a properly computed dlfs_avail.

* Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv.
  This prevents a panic, but, if the cleaner is feeding the filesystem
  the wrong data, you are still in a world of hurt.

* Cleanup: remove explicit references of DEV_BSIZE in favor of
  btodb()/dbtob().

lfs_cleanerd:

* Make -n mean "send N segments' blocks through a single call to
  lfs_markv".  Previously it had meant "clean N segments though N calls
  to lfs_markv, before looking again to see if more need to be cleaned".
  The new behavior gives better packing of direct data on disk with as
  little metadata as possible, largely alleviating the problem that the
  cleaner can consume more disk through inefficient use of metadata than
  it frees by moving dirty data away from clean "holes" to produce
  entirely clean segments.

* Make -b mean "read as many segments as necessary to write N segments
  of dirty data back to disk", rather than its former meaning of "read
  as many segments as necessary to free N segments worth of space".  The
  new meaning, combined with the new -n behavior described above,
  further aids in cleaning storage efficiency as entire segments can be
  written at once, using as few blocks as possible for segment summaries
  and inode blocks.

* Make the cleaner take note of segments which could not be cleaned due
  to error, and not attempt to clean them until they are entirely free
  of dirty blocks.  This prevents the case in which a cleanerd running
  with -n 1 and without -b (formerly the default) would spin trying
  repeatedly to clean a corrupt segment, while the remaining space
  filled and deadlocked the filesystem.

* Update the lfs_cleanerd manual page to describe all the options,
  including the changes mentioned here (in particular, the -b and -n
  flags were previously undocumented).

fsck_lfs:

* Check, and optionally fix, lfs_avail (to an exact figure) and
  lfs_bfree (within a margin of error) in pass 5.

newfs_lfs:

* Reduce the default dlfs_minfreeseg to 1/20 of the total segments.

* Add a warning if the sgs disklabel field is 16 (the default for FFS'
  cpg, but not usually desirable for LFS' sgs: 5--8 is a better range).

* Change the calculation of lfs_avail and lfs_bfree, corresponding to
  the kernel changes mentioned above.

mount_lfs:

* Add -N and -b options to pass corresponding -n and -b options to
  lfs_cleanerd.

* Default to calling lfs_cleanerd with "-b -n 4".


[All of these changes were largely tested in the 1.5 branch, with the
idea that they (along with previous un-pulled-up work) could be applied
to the branch while it was still in ALPHA2; however my test system has
experienced corruption on another filesystem (/dev/console has gone
missing :^), and, while I believe this unrelated to the LFS changes, I
cannot with good conscience request that the changes be pulled up.]
2000-09-09 04:49:54 +00:00
perseant f184685d10 cleaner changes corrseponding to kernel changes 2000-07-04 22:36:17 +00:00
perseant 9a38f49c57 User-level changes corrseponding to my latest kernel changes.
newfs_lfs gives lfs_minfreeseg a value of 1/8 of the total segments on
the disk, based on rough empirical data, but this should be refined in
the future.
2000-07-03 01:49:11 +00:00
perseant bbc8485d45 Make sure to segunmap segments on error in lfs_bmapv or lfs_markv. Prevents
a memory leak of by default 1 Mb per error.  May fix PR #9149.
2000-06-21 01:58:52 +00:00
perseant d244493927 Take care of memory leaks 2000-01-18 08:02:30 +00:00
perseant fcb8440e38 If the child cleaner dies repeatedly without doing anything, give up. Uses
similar logic to inetd to identify such looping.
1999-11-09 20:33:37 +00:00
perseant 2750b8620f Make datobyte do its arithmetic explicitly in 64 bits, so that segments
beyond the first 2G of disk can be cleaned.
1999-11-09 01:06:39 +00:00
perseant 64846ce5dc Don't complain that we can't fstat the Ifile, if it's because the filesystem
has been unmounted.  (I.e., don't give errors every time an LFS is unmounted.)
1999-10-07 18:26:58 +00:00
soren 30d18a154c Remove extraneous colons and newlines from perror(3) arguments. 1999-09-30 12:35:52 +00:00
perseant 193a2fa9f0 Remove partial-segment timestamp optimization from lfs_segmapv. If "time"
is not monotonically increasing (e.g. clock is slaved to another system)
the optimization will result in segments being treated as corrupt
(uncleanable).  If enough such "bad" segments were created, the cleaner would
clean continuously, and after some time the system would panic with "no
clean segments".

(Legitimately old partial-segments are relatively rare, and will have their
blocks culled by lfs_bmapv.)
1999-08-25 00:14:13 +00:00
tron cf71eb8d35 Correct wrong conversion specifications in calls to syslog(3). 1999-06-16 16:34:29 +00:00
perseant 0486add127 The cleaner now marks empty segments clean without having to read their
contents, a substantial optimization if the work load is right: if enough
empty segments are available, the cleaner never has to read or write *any*
blocks except those on the Ifile.  When the cleaner wakes up it marks all
empty segments clean before deciding whether any further segments need to
be cleaned.

Fixed overflow bugs in the cleaner's handling of the cost/benefit metric
for empty segments.
1999-06-15 22:33:48 +00:00
garbled f124765044 More and more .Os cleanups. .Os is defined in the tmac.doc-common file,
so we shouldn't override it with versions in the manpages.  Many more to
come.
1999-03-22 18:25:43 +00:00
drochner 8f59d4063a work around egcs prints format warning on alpha
(XXX bad fix, but egcs doesn't like a "quad_t" here)
1999-03-14 11:43:25 +00:00
drochner 8ae646e9ae work around printf format warnings on alpha 1999-03-14 11:39:28 +00:00
perseant 2c952176c1 Cleaner changes corresponding to in-kernel LFS changes. In particular, the
cleaner understands fragments; and it knows to change bi_bp to reflect a
change in bi_daddr, if lfs_bmapv says that there is one.
1999-03-10 00:57:16 +00:00
ross ee65bcaa99 Cast quad_t and off_t objects to (long long) for printing with %q.
(Fix a sort-of-LP64 egcs printf warning.)

It's unfortunate that off_t and quad_t don't print with %q. I wonder
what would happen if alpha changed these from long -> long long? It's
the same actual size in bits either way.
1998-10-15 00:29:51 +00:00
christos 977a282bd1 PR/6248: Konrad Schroder: lfs_cleanerd could use syslog() for notices,
instead of err()/warn()
1998-10-07 15:00:34 +00:00
christos e044377a39 PR/6249: Konrad Schroder lfs_cleanerd uses wrong units when computing which
segments to clean
1998-10-07 14:57:30 +00:00
christos 2e885feacf PR/6247: Konrad Schroder: lfs_cleanerd overflows char register counting live
bytes
1998-10-07 14:56:22 +00:00
pk 547e652f5d type fixes, in part from PR#6032 1998-09-11 21:21:29 +00:00
perry bfd526219d bzero->memset, bcopy->memcpy, bcmp->memcmp 1998-08-10 02:57:23 +00:00
hubertf 80a44df7c2 use correct getopt parameters, as per PR 5480 from Konrad Schroder <perseant@hhhh.org> 1998-05-22 02:21:00 +00:00
kleink 2efb2aaff6 Need <time.h> for ctime() prototype. 1998-04-01 14:39:56 +00:00
kleink d868740f80 Need <time.h> for time() and ctime() prototypes. 1998-04-01 14:37:45 +00:00
fvdl e5bc90f40c Merge with Lite2 + local changes 1998-03-01 02:20:01 +00:00
mycroft 4462053ab3 Fill in missing (default) mmap(2) flags. 1998-02-20 09:27:18 +00:00
perry 6982982f8d macroize BSD, NetBSD, FreeBSD and misc cleanup 1998-02-06 05:39:31 +00:00
lukem dc3678728d use CPPFLAGS instead of CFLAGS 1997-10-22 05:45:08 +00:00
enami 7eda8e3153 Fix usage of .Nm. 1997-10-20 02:05:33 +00:00
enami 80b4fe5249 No need to decalre optind. 1997-10-20 02:04:03 +00:00
mrg 2268d2dfb3 merge lite-2 Makefiles (rcsids), and turn on WARNS for all of libexec. 1997-10-08 09:07:11 +00:00
enami 42eb13e478 Compare a return value of getopt() against -1 instead of EOF. 1997-10-08 00:56:46 +00:00
mrg ad681f7eae WARNS?=1. RCS ids. 1997-10-07 13:44:03 +00:00
mrg 3d8d628a25 WARNS?=1. RCS ids. 1997-10-07 13:39:56 +00:00