Aren
/
NetBSD
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
NetBSD/sbin/fsck_lfs/pass5.c

185 lines
5.9 KiB
C
Raw Permalink Normal View History

Avoid common symbols for fsck_lfs.
2020-04-03 22:36:32 +03:00
/* $NetBSD: pass5.c,v 1.37 2020/04/03 19:36:33 joerg Exp $ */
Convert to NetBSD source code style
2000-05-23 05:48:52 +04:00
/*-
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
* Copyright (c) 2000, 2003 The NetBSD Foundation, Inc.
Convert to NetBSD source code style
2000-05-23 05:48:52 +04:00
* All rights reserved.
*
* This code is derived from software contributed to The NetBSD Foundation
* by Konrad E. Schroder <perseant@hhhh.org>.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
* ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
* TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
* BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
* CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
* SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
* INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
* CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
* ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
* POSSIBILITY OF SUCH DAMAGE.
*/
Initial checkin of fsck_lfs. This version cannot do any repair (-p flag does nothing, and one of -p or -n is required) but can be useful as a diagnostic tool.
1999-03-18 05:02:18 +03:00
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
#include <sys/types.h>
Initial checkin of fsck_lfs. This version cannot do any repair (-p flag does nothing, and one of -p or -n is required) but can be useful as a diagnostic tool.
1999-03-18 05:02:18 +03:00
#include <sys/param.h>
#include <sys/time.h>
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
#include <sys/buf.h>
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
#include <sys/mount.h>
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
Cleanups and hacks to make lfs userland stuff build: - lfs_cksum.c doesn't actually need ulfs_inode.h any more. - neither does lfs_itimes.c. - add hacks to fsck_lfs to make it compile. - add hacks to newfs_lfs to make it compile. - fix warning in ulfs_quota.c when quotas are fully disabled (as I guess is happening with the rumpity version) XXX: This commit adds -I${NETBSDSRCDIR}/sys to the Makefiles for XXX: fsck_lfs, newfs_lfs, and lfs_cleanerd. This needs to be cleaned XXX: up ASAP; but I consider this less problematic in the short term XXX: than spewing ulfs_*.h into /usr/include.
2013-06-06 04:54:49 +04:00
#define vnode uvnode
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
#include <ufs/lfs/lfs.h>
Add a new lfs header file: lfs_accessors.h. This contains all the accessor functions and macros out of lfs.h. Add an include of lfs_accessors.h after all uses of lfs.h... except for code that wants to define its own struct lfs-alike that the accessors are supposed to play along with. For these, set STRUCT_LFS and include lfs_accessors.h after the necessary structure has been defined, so that lfs_accessors.h can emit functions in terms of it.
2015-07-28 08:09:34 +03:00
#include <ufs/lfs/lfs_accessors.h>
Tidy up the LFS userland build hacks. Don't use -I${NETBSDSRCDIR}/sys; don't include files other than the exported LFS headers, which are lfs.h, lfs_inode.h, and (for now) lfs_extern.h.
2013-06-08 06:16:03 +04:00
#include <ufs/lfs/lfs_inode.h>
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
#undef vnode
Initial checkin of fsck_lfs. This version cannot do any repair (-p flag does nothing, and one of -p or -n is required) but can be useful as a diagnostic tool.
1999-03-18 05:02:18 +03:00
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
#include <string.h>
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
#include "bufcache.h"
rename lfs.h to lfs_user.h so that it does not conflict.
2005-09-13 08:14:17 +04:00
#include "lfs_user.h"
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
Initial checkin of fsck_lfs. This version cannot do any repair (-p flag does nothing, and one of -p or -n is required) but can be useful as a diagnostic tool.
1999-03-18 05:02:18 +03:00
#include "fsck.h"
#include "extern.h"
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
#include "fsutil.h"
Initial checkin of fsck_lfs. This version cannot do any repair (-p flag does nothing, and one of -p or -n is required) but can be useful as a diagnostic tool.
1999-03-18 05:02:18 +03:00
void
ANSIfy, WARNS=2
2005-01-19 22:41:59 +03:00
pass5(void)
Initial checkin of fsck_lfs. This version cannot do any repair (-p flag does nothing, and one of -p or -n is required) but can be useful as a diagnostic tool.
1999-03-18 05:02:18 +03:00
{
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
SEGUSE *su;
pass5: dereference of an uninitialized pointer.
2004-05-14 14:41:12 +04:00
struct ubuf *bp;
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
int i;
Fix assorted 64 -> 32 truncations in lfs. Also, some minor tidyups and corrections in passing.
2015-08-02 21:08:12 +03:00
daddr_t bb; /* total number of used blocks (lower bound) */
daddr_t ubb; /* upper bound number of used blocks */
daddr_t avail; /* blocks available for writing */
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
unsigned long dmeta; /* blocks in segsums and inodes */
int nclean; /* clean segments */
size_t labelskew;
int diddirty;
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
/*
* Check segment holdings against actual holdings. Check for
Check parts of pass 5 even if only rolling forward. We can't check the true segment holdings against the blocks held by the inodes, but we can still check the cleanerinfo data against the segment table.
2005-04-24 00:21:03 +04:00
* "clean" segments that contain live data. If we are only
* rolling forward, we can't check the segment holdings, but
* we can still check the cleanerinfo data.
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
*/
Report, and detect and correct inconsistencies in, the number of clean segments. Patches from Jesse Off <joff@gci-net.com> (PR #11470).
2000-11-13 03:30:48 +03:00
nclean = 0;
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
avail = 0;
bb = ubb = 0;
Check/fix accounting of lfs_dmeta. Patch from Jesse Off <joff@gci-net.com> (PR #11534).
2000-11-21 09:24:26 +03:00
dmeta = 0;
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
for (i = 0; i < lfs_sb_getnseg(fs); i++) {
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
diddirty = 0;
LFS_SEGENTRY(su, fs, i, bp);
Check parts of pass 5 even if only rolling forward. We can't check the true segment holdings against the blocks held by the inodes, but we can still check the cleanerinfo data against the segment table.
2005-04-24 00:21:03 +04:00
if (!preen && !(su->su_flags & SEGUSE_DIRTY) &&
Convert to NetBSD source code style
2000-05-23 05:48:52 +04:00
seg_table[i].su_nbytes > 0) {
Several fixes to improve the reliability of the roll-forward agent. Also, note "properly orphaned" files as distinct from corrupted files.
2006-09-01 23:52:48 +04:00
pwarn("CLEAN SEGMENT %d CONTAINS %d BYTES\n",
Various improvements to fsck_lfs, to wit: * Add lfs_balloc capability to the lfs library. * Extend the Ifile if we run out of free inodes when creating lost+found. * Don't roll forward if we have allocated a lost+found, to avoid conflicts when adding new files in roll-forward. * Make some messages slightly more verbose (e.g. include inode number, and use pwarn() instead of printf() so the messages include the device name when preening). * Change superblock detection/avoidance to use the offset table in the primary superblock, rather than looking at the contents. * Be more verbose about various operations when passed the -d flag, especially roll-forward. * Be more careful about dirops during roll forward, since the cleaner can sometimes write blocks from dirop vnodes. Detect and avoid this problem. * Always check the free list, even if given -i; if we're going to write it we have to check it first. * Mark inodes dirty when blocks are found during roll forward, so the inodes are written with the new block locations. * Update size of inodes if blocks beyond EOF are found during roll forward. * Fix segment accounting for blocks and inodes found during roll forward. * Report statistics on roll forward: how many new/deleted/moved files and how many updated blocks (or "nothing new"). * Don't care if the device being checked is really a device, if we have been passed the -f flag (to facilitate automated testing). * When writing to the disk, use the current time in the segment headers rathern than time 0. * When passed the -i flag, locate the partial segment containing the Ifile inode and use that to calculate lfs_offset, lfs_curseg, lfs_nextseg. (Again for automated testing.)
2006-07-19 03:37:13 +04:00
i, seg_table[i].su_nbytes);
Remove the free list ordering/disordering code, since the kernel now keeps the list in order (ordering it on mount). Regularize error messages: these are now all in ALL CAPS, with all hex numbers (not reported in caps) prefixed by 0x. (The non-fsck-specific messages are an exception to this all-caps rule.)
2006-04-17 23:05:16 +04:00
if (reply("MARK SEGMENT DIRTY")) {
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
su->su_flags |= SEGUSE_DIRTY;
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
++diddirty;
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
}
}
Several fixes to improve the reliability of the roll-forward agent. Also, note "properly orphaned" files as distinct from corrupted files.
2006-09-01 23:52:48 +04:00
if (!preen && su->su_nbytes != seg_table[i].su_nbytes) {
Remove the free list ordering/disordering code, since the kernel now keeps the list in order (ordering it on mount). Regularize error messages: these are now all in ALL CAPS, with all hex numbers (not reported in caps) prefixed by 0x. (The non-fsck-specific messages are an exception to this all-caps rule.)
2006-04-17 23:05:16 +04:00
pwarn("SEGMENT %d CLAIMS %d BYTES BUT HAS %d",
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
i, su->su_nbytes, seg_table[i].su_nbytes);
if ((int32_t)su->su_nbytes >
(int32_t)seg_table[i].su_nbytes)
Remove the free list ordering/disordering code, since the kernel now keeps the list in order (ordering it on mount). Regularize error messages: these are now all in ALL CAPS, with all hex numbers (not reported in caps) prefixed by 0x. (The non-fsck-specific messages are an exception to this all-caps rule.)
2006-04-17 23:05:16 +04:00
pwarn(" (HIGH BY %d)\n", su->su_nbytes -
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
seg_table[i].su_nbytes);
Check for cycles in the inode free list, and for free inodes not on the free list.
2000-05-30 08:33:14 +04:00
else
Remove the free list ordering/disordering code, since the kernel now keeps the list in order (ordering it on mount). Regularize error messages: these are now all in ALL CAPS, with all hex numbers (not reported in caps) prefixed by 0x. (The non-fsck-specific messages are an exception to this all-caps rule.)
2006-04-17 23:05:16 +04:00
pwarn(" (LOW BY %d)\n", -su->su_nbytes +
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
seg_table[i].su_nbytes);
Remove the free list ordering/disordering code, since the kernel now keeps the list in order (ordering it on mount). Regularize error messages: these are now all in ALL CAPS, with all hex numbers (not reported in caps) prefixed by 0x. (The non-fsck-specific messages are an exception to this all-caps rule.)
2006-04-17 23:05:16 +04:00
if (reply("FIX")) {
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
su->su_nbytes = seg_table[i].su_nbytes;
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
++diddirty;
fsck_lfs can now write to the filesystem, allowing it to correct most (though still not all) errors in a damaged lfs. Segment byte accounting is corrected in pass 5. "fsck_lfs -p" will do a partial roll-forward, verifying the checkpoint from the newer superblock. fscknames[] is updated so that fsck knows about fsck_lfs.
2000-05-16 08:55:58 +04:00
}
}
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
if (su->su_flags & SEGUSE_DIRTY) {
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs. This was done so that boot blocks that want to compile both FFS and LFS in the same file work.
2013-06-18 22:18:57 +04:00
bb += lfs_btofsb(fs, su->su_nbytes +
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
su->su_nsums * lfs_sb_getsumsize(fs));
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs. This was done so that boot blocks that want to compile both FFS and LFS in the same file work.
2013-06-18 22:18:57 +04:00
ubb += lfs_btofsb(fs, su->su_nbytes +
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
su->su_nsums * lfs_sb_getsumsize(fs) +
su->su_ninos * lfs_sb_getibsize(fs));
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs. This was done so that boot blocks that want to compile both FFS and LFS in the same file work.
2013-06-18 22:18:57 +04:00
dmeta += lfs_btofsb(fs,
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
lfs_sb_getsumsize(fs) * su->su_nsums);
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs. This was done so that boot blocks that want to compile both FFS and LFS in the same file work.
2013-06-18 22:18:57 +04:00
dmeta += lfs_btofsb(fs,
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
lfs_sb_getibsize(fs) * su->su_ninos);
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
} else {
Report, and detect and correct inconsistencies in, the number of clean segments. Patches from Jesse Off <joff@gci-net.com> (PR #11470).
2000-11-13 03:30:48 +03:00
nclean++;
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs. This was done so that boot blocks that want to compile both FFS and LFS in the same file work.
2013-06-18 22:18:57 +04:00
avail += lfs_segtod(fs, 1);
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
if (su->su_flags & SEGUSE_SUPERBLOCK)
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs. This was done so that boot blocks that want to compile both FFS and LFS in the same file work.
2013-06-18 22:18:57 +04:00
avail -= lfs_btofsb(fs, LFS_SBPAD);
Use accessor functions for the version field of the lfs superblock. I thought at first maybe the cases that test the version should be rolled into the accessors, but on the whole I think the conclusion on that is no.
2015-08-02 21:14:16 +03:00
if (i == 0 && lfs_sb_getversion(fs) > 1 &&
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
lfs_sb_gets0addr(fs) < lfs_btofsb(fs, LFS_LABELPAD))
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs. This was done so that boot blocks that want to compile both FFS and LFS in the same file work.
2013-06-18 22:18:57 +04:00
avail -= lfs_btofsb(fs, LFS_LABELPAD) -
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
lfs_sb_gets0addr(fs);
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
}
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
if (diddirty)
VOP_BWRITE(bp);
else
Give brelse() a second argument so that it matches the kernel. fsck_lfs now compiles again.
2007-10-09 01:39:49 +04:00
brelse(bp, 0);
Initial checkin of fsck_lfs. This version cannot do any repair (-p flag does nothing, and one of -p or -n is required) but can be useful as a diagnostic tool.
1999-03-18 05:02:18 +03:00
}
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
/* Also may be available bytes in current seg */
Switch to accessor functions for elements of the LFS on-disk superblock. This will allow switching between 32/64 bit forms on the fly; it will also allow handling LFS_EI reasonably tidily. (That currently doesn't work on the superblock.) It also gets rid of cpp abuse in the form of fake structure member macros. Also, instead of doing sleep/wakeup on &lfs_avail and &lfs_nextseg inside the on-disk superblock, add extra elements to the in-memory struct lfs for this. (XXX: these should be changed to condvars, but not right now) XXX: this migrates a structure needed by the lfs code in libsa (struct salfs) into lfs.h, where it doesn't belong, but for the time being this is necessary in order to allow the accessors (and the various lfs macros and other goop that relies on them) to compile.
2015-07-24 09:56:41 +03:00
i = lfs_dtosn(fs, lfs_sb_getoffset(fs));
avail += lfs_sntod(fs, i + 1) - lfs_sb_getoffset(fs);
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
/* But do not count minfreesegs */
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
avail -= lfs_segtod(fs, (lfs_sb_getminfreeseg(fs) -
(lfs_sb_getminfreeseg(fs) / 2)));
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
/* Note we may have bytes to write yet */
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs. This was done so that boot blocks that want to compile both FFS and LFS in the same file work.
2013-06-18 22:18:57 +04:00
avail -= lfs_btofsb(fs, locked_queue_bytes);
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
Be more efficient with the hash tables for the buffer and vnode caches. Note that roll-forward can add more inodes to the filesystem; don't overflow the tables but reallocate them.
2005-04-12 03:19:24 +04:00
if (idaddr)
Check parts of pass 5 even if only rolling forward. We can't check the true segment holdings against the blocks held by the inodes, but we can still check the cleanerinfo data against the segment table.
2005-04-24 00:21:03 +04:00
pwarn("NOTE: when using -i, expect discrepancies in dmeta,"
" avail, nclean, bfree\n");
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
if (dmeta != lfs_sb_getdmeta(fs)) {
pwarn("DMETA GIVEN AS %d, SHOULD BE %ld\n",
lfs_sb_getdmeta(fs), dmeta);
Remove the free list ordering/disordering code, since the kernel now keeps the list in order (ordering it on mount). Regularize error messages: these are now all in ALL CAPS, with all hex numbers (not reported in caps) prefixed by 0x. (The non-fsck-specific messages are an exception to this all-caps rule.)
2006-04-17 23:05:16 +04:00
if (preen || reply("FIX")) {
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
lfs_sb_setdmeta(fs, dmeta);
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
sbdirty();
}
Check/fix accounting of lfs_dmeta. Patch from Jesse Off <joff@gci-net.com> (PR #11534).
2000-11-21 09:24:26 +03:00
}
Switch to accessor functions for elements of the LFS on-disk superblock. This will allow switching between 32/64 bit forms on the fly; it will also allow handling LFS_EI reasonably tidily. (That currently doesn't work on the superblock.) It also gets rid of cpp abuse in the form of fake structure member macros. Also, instead of doing sleep/wakeup on &lfs_avail and &lfs_nextseg inside the on-disk superblock, add extra elements to the in-memory struct lfs for this. (XXX: these should be changed to condvars, but not right now) XXX: this migrates a structure needed by the lfs code in libsa (struct salfs) into lfs.h, where it doesn't belong, but for the time being this is necessary in order to allow the accessors (and the various lfs macros and other goop that relies on them) to compile.
2015-07-24 09:56:41 +03:00
if (avail != lfs_sb_getavail(fs)) {
Use intmax_t to print daddr_t; from sevan@.
2015-08-03 07:16:10 +03:00
pwarn("AVAIL GIVEN AS %jd, SHOULD BE %jd\n",
(intmax_t)lfs_sb_getavail(fs), (intmax_t)avail);
Remove the free list ordering/disordering code, since the kernel now keeps the list in order (ordering it on mount). Regularize error messages: these are now all in ALL CAPS, with all hex numbers (not reported in caps) prefixed by 0x. (The non-fsck-specific messages are an exception to this all-caps rule.)
2006-04-17 23:05:16 +04:00
if (preen || reply("FIX")) {
Switch to accessor functions for elements of the LFS on-disk superblock. This will allow switching between 32/64 bit forms on the fly; it will also allow handling LFS_EI reasonably tidily. (That currently doesn't work on the superblock.) It also gets rid of cpp abuse in the form of fake structure member macros. Also, instead of doing sleep/wakeup on &lfs_avail and &lfs_nextseg inside the on-disk superblock, add extra elements to the in-memory struct lfs for this. (XXX: these should be changed to condvars, but not right now) XXX: this migrates a structure needed by the lfs code in libsa (struct salfs) into lfs.h, where it doesn't belong, but for the time being this is necessary in order to allow the accessors (and the various lfs macros and other goop that relies on them) to compile.
2015-07-24 09:56:41 +03:00
lfs_sb_setavail(fs, avail);
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
sbdirty();
}
}
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
if (nclean != lfs_sb_getnclean(fs)) {
pwarn("NCLEAN GIVEN AS %d, SHOULD BE %d\n", lfs_sb_getnclean(fs),
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
nclean);
Remove the free list ordering/disordering code, since the kernel now keeps the list in order (ordering it on mount). Regularize error messages: these are now all in ALL CAPS, with all hex numbers (not reported in caps) prefixed by 0x. (The non-fsck-specific messages are an exception to this all-caps rule.)
2006-04-17 23:05:16 +04:00
if (preen || reply("FIX")) {
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
lfs_sb_setnclean(fs, nclean);
Report, and detect and correct inconsistencies in, the number of clean segments. Patches from Jesse Off <joff@gci-net.com> (PR #11470).
2000-11-13 03:30:48 +03:00
sbdirty();
}
}
Add working writing ability to fsck_lfs, including roll-forward, based on a partial-segment writer ported from the kernel.
2003-03-28 11:09:52 +03:00
Make the "-O" (start filesystem offset) flag to newfs_lfs work correctly, and update fsck_lfs and dumplfs to deal with it. Note that while the argument to -O is given in disk sectors, it must be a multiple of the fragment size, and although it can be lower than the label or superblock, it can't intersect either.
2003-02-23 07:32:05 +03:00
labelskew = 0;
Use accessor functions for the version field of the lfs superblock. I thought at first maybe the cases that test the version should be rolled into the accessors, but on the whole I think the conclusion on that is no.
2015-08-02 21:14:16 +03:00
if (lfs_sb_getversion(fs) > 1 &&
More lfs superblock accessors. (This changes the rest of the code over; all the accessors were already added.) The difference between this commit and the previous one is arbitrary, but the previous one passed the regression tests on its own so I'm keeping it separate to help with any bisections that might be needed in the future.
2015-07-24 09:59:31 +03:00
lfs_sb_gets0addr(fs) < lfs_btofsb(fs, LFS_LABELPAD))
Prefix most of the cpp macros with lfs_ and LFS_ to avoid conflicts with ffs. This was done so that boot blocks that want to compile both FFS and LFS in the same file work.
2013-06-18 22:18:57 +04:00
labelskew = lfs_btofsb(fs, LFS_LABELPAD);
Switch to accessor functions for elements of the LFS on-disk superblock. This will allow switching between 32/64 bit forms on the fly; it will also allow handling LFS_EI reasonably tidily. (That currently doesn't work on the superblock.) It also gets rid of cpp abuse in the form of fake structure member macros. Also, instead of doing sleep/wakeup on &lfs_avail and &lfs_nextseg inside the on-disk superblock, add extra elements to the in-memory struct lfs for this. (XXX: these should be changed to condvars, but not right now) XXX: this migrates a structure needed by the lfs code in libsa (struct salfs) into lfs.h, where it doesn't belong, but for the time being this is necessary in order to allow the accessors (and the various lfs macros and other goop that relies on them) to compile.
2015-07-24 09:56:41 +03:00
if (lfs_sb_getbfree(fs) > lfs_sb_getdsize(fs) - bb - labelskew ||
lfs_sb_getbfree(fs) < lfs_sb_getdsize(fs) - ubb - labelskew) {
Fix assorted 64 -> 32 truncations in lfs. Also, some minor tidyups and corrections in passing.
2015-08-02 21:08:12 +03:00
pwarn("BFREE GIVEN AS %jd, SHOULD BE BETWEEN %jd AND %jd\n",
Switch to accessor functions for elements of the LFS on-disk superblock. This will allow switching between 32/64 bit forms on the fly; it will also allow handling LFS_EI reasonably tidily. (That currently doesn't work on the superblock.) It also gets rid of cpp abuse in the form of fake structure member macros. Also, instead of doing sleep/wakeup on &lfs_avail and &lfs_nextseg inside the on-disk superblock, add extra elements to the in-memory struct lfs for this. (XXX: these should be changed to condvars, but not right now) XXX: this migrates a structure needed by the lfs code in libsa (struct salfs) into lfs.h, where it doesn't belong, but for the time being this is necessary in order to allow the accessors (and the various lfs macros and other goop that relies on them) to compile.
2015-07-24 09:56:41 +03:00
(intmax_t)lfs_sb_getbfree(fs),
Fix assorted 64 -> 32 truncations in lfs. Also, some minor tidyups and corrections in passing.
2015-08-02 21:08:12 +03:00
(intmax_t)(lfs_sb_getdsize(fs) - ubb - labelskew),
(intmax_t)(lfs_sb_getdsize(fs) - bb - labelskew));
Remove the free list ordering/disordering code, since the kernel now keeps the list in order (ordering it on mount). Regularize error messages: these are now all in ALL CAPS, with all hex numbers (not reported in caps) prefixed by 0x. (The non-fsck-specific messages are an exception to this all-caps rule.)
2006-04-17 23:05:16 +04:00
if (preen || reply("FIX")) {
Switch to accessor functions for elements of the LFS on-disk superblock. This will allow switching between 32/64 bit forms on the fly; it will also allow handling LFS_EI reasonably tidily. (That currently doesn't work on the superblock.) It also gets rid of cpp abuse in the form of fake structure member macros. Also, instead of doing sleep/wakeup on &lfs_avail and &lfs_nextseg inside the on-disk superblock, add extra elements to the in-memory struct lfs for this. (XXX: these should be changed to condvars, but not right now) XXX: this migrates a structure needed by the lfs code in libsa (struct salfs) into lfs.h, where it doesn't belong, but for the time being this is necessary in order to allow the accessors (and the various lfs macros and other goop that relies on them) to compile.
2015-07-24 09:56:41 +03:00
lfs_sb_setbfree(fs,
((lfs_sb_getdsize(fs) - labelskew - ubb) +
lfs_sb_getdsize(fs) - labelskew - bb) / 2);
Various bug-fixes to LFS, to wit: Kernel: * Add runtime quantity lfs_ravail, the number of disk-blocks reserved for writing. Writes to the filesystem first reserve a maximum amount of blocks before their write is allowed to proceed; after the blocks are allocated the reserved total is reduced by a corresponding amount. If the lfs_reserve function cannot immediately reserve the requested number of blocks, the inode is unlocked, and the thread sleeps until the cleaner has made enough space available for the blocks to be reserved. In this way large files can be written to the filesystem (or, smaller files can be written to a nearly-full but thoroughly clean filesystem) and the cleaner can still function properly. * Remove explicit switching on dlfs_minfreeseg from the kernel code; it is now merely a fs-creation parameter used to compute dlfs_avail and dlfs_bfree (and used by fsck_lfs(8) to check their accuracy). Its former role is better assumed by a properly computed dlfs_avail. * Bounds-check inode numbers submitted through lfs_bmapv and lfs_markv. This prevents a panic, but, if the cleaner is feeding the filesystem the wrong data, you are still in a world of hurt. * Cleanup: remove explicit references of DEV_BSIZE in favor of btodb()/dbtob(). lfs_cleanerd: * Make -n mean "send N segments' blocks through a single call to lfs_markv". Previously it had meant "clean N segments though N calls to lfs_markv, before looking again to see if more need to be cleaned". The new behavior gives better packing of direct data on disk with as little metadata as possible, largely alleviating the problem that the cleaner can consume more disk through inefficient use of metadata than it frees by moving dirty data away from clean "holes" to produce entirely clean segments. * Make -b mean "read as many segments as necessary to write N segments of dirty data back to disk", rather than its former meaning of "read as many segments as necessary to free N segments worth of space". The new meaning, combined with the new -n behavior described above, further aids in cleaning storage efficiency as entire segments can be written at once, using as few blocks as possible for segment summaries and inode blocks. * Make the cleaner take note of segments which could not be cleaned due to error, and not attempt to clean them until they are entirely free of dirty blocks. This prevents the case in which a cleanerd running with -n 1 and without -b (formerly the default) would spin trying repeatedly to clean a corrupt segment, while the remaining space filled and deadlocked the filesystem. * Update the lfs_cleanerd manual page to describe all the options, including the changes mentioned here (in particular, the -b and -n flags were previously undocumented). fsck_lfs: * Check, and optionally fix, lfs_avail (to an exact figure) and lfs_bfree (within a margin of error) in pass 5. newfs_lfs: * Reduce the default dlfs_minfreeseg to 1/20 of the total segments. * Add a warning if the sgs disklabel field is 16 (the default for FFS' cpg, but not usually desirable for LFS' sgs: 5--8 is a better range). * Change the calculation of lfs_avail and lfs_bfree, corresponding to the kernel changes mentioned above. mount_lfs: * Add -N and -b options to pass corresponding -n and -b options to lfs_cleanerd. * Default to calling lfs_cleanerd with "-b -n 4". [All of these changes were largely tested in the 1.5 branch, with the idea that they (along with previous un-pulled-up work) could be applied to the branch while it was still in ALPHA2; however my test system has experienced corruption on another filesystem (/dev/console has gone missing :^), and, while I believe this unrelated to the LFS changes, I cannot with good conscience request that the changes be pulled up.]
2000-09-09 08:49:54 +04:00
sbdirty();
}
}
Initial checkin of fsck_lfs. This version cannot do any repair (-p flag does nothing, and one of -p or -n is required) but can be useful as a diagnostic tool.
1999-03-18 05:02:18 +03:00
}